rank-1 matrix

Rank-one Matrix

Let $A\in {\mathbb{R}}^{m\times n}$. If there exist two vectors $x\in {\mathbb{R}}^m$ and $y\in {\mathbb{R}}^n$ such that

$$A=x{y}^T$$

then we call $A$ a rank-one matrix.

This is equivalent to saying that the column space (or row space) of $A$ is spanned by a single vector.

SVD as Sum of Outer Products

The SVD can be written as a sum of ${\mathbb{R}}^{m\times m}$ rank-1 matrices formed by outer products of corresponding singular vectors, weighted by singular values:

$$\begin{array}{rl}X& =U\Sigma V^T\\ & ={\sigma }_1{u}_1{v}_1^T+{\sigma }_2{u}_2{v}_2^T+\cdots +{\sigma }_m{u}_m{v}_m^T+0\\ & ={\sigma }_1\left[\begin{array}{c}\\ u_1\\ \end{array}\right]\left[\begin{array}{ccc}& v_1^T& \end{array}\right]+{\sigma }_2\left[\begin{array}{c}\\ u_2\\ \end{array}\right]\left[\begin{array}{ccc}& v_2^T& \end{array}\right]+\cdots +{\sigma }_m\left[\begin{array}{c}\\ u_m\\ \end{array}\right]\left[\begin{array}{ccc}& v_m^T& \end{array}\right]\end{array}$$

Each term ${\sigma }_k{u}_k{v}_k^T$ is a rank-1 matrix by construction, fully depending on / explained by one row and col, and captures a fundamental (orthogonal) direction of variation in the data, weighted by the corresponding singular value ${\sigma }_k$.

This sum of rank-1 matrices increasingly improves the approximation of $X$ (like denoising! ²).

Link to original