Hey guys! I took a linear algebra course last year, and now just as self study, I want to get into some less surface-level linalg topics that weren't covered as part of the course I took.
Currently, I'm trying to understand singular value decomp, and I've got just a couple questions, mainly about the geometric intuition bout things.
- I get that the matrices A^T A and A A^T describe how the rows and columns of A correlate with one another, and that finding the axes of maximum variance gives you the most dominant features in this space. I don't understand, however, what the eigenvectors of these two matrices mean geometrically/intuitively, and why using these eigenvectors/values (ranked by the eigenvalue size) gives you these principal directions of variation.
- Why do the singular values of matrix A happen to be the square roots of the eigenvalues of A^T A and A A^T?
- The idea of eigenvectors seems to be to find the axes within which direction is unchanged, and the point of singular vectors seems to be to extend this idea to non-square matrices. But singular vectors don't preserve direction like eigenvectors do, as a result of non-square matrices having different input/output space dimensions, so what do they preserve? I hear that they preserve orthogonality, but I don't really understand what that means geometrically/practically.
- This is more of a general question about transposes in general. Say a matrix applies some sort of transformation, what will the transpose of that matrix apply as a transformation? What about the A^T A and A A^T matrices - what type of transformations will they apply? Or do transposes not really have a geometric meaning with respect to their original matrix's geometric transformation?
Thanks!