second moment matrix

Second Moment Matrix

The second moment matrix $M\in {\mathbb{R}}^{d\times d}$ represents the feature-feature relationships of samples $\mathbf{x}\in {\mathbb{R}}^{1\times d}$ by taking the average outer product:

$$M=\mathbb{E}[{\mathbf{x}}^T\mathbf{x}]$$

• The empirical estimator is $\hat{M}=\frac{1}{n}{X}^{T}X$, where $X\in {\mathbb{R}}^{n\times d}$
• Entries $(i,j)$ are the mean dot products of the feature columns: ${\hat{M}}_{i,j}=\frac{1}{n}{\sum }_k{x}_{k,i}{x}_{k,j}$, so the feature-feature relationships
• Entries $(i,i)$ on the diagonal are the mean squares of the feature columns: ${\hat{M}}_{i,i}=\frac{1}{n}{\sum }_k{x}_{k,i}^2$, so the second moments.
• Is the feature gram matrix scaled by $\frac{1}{n}$.
• If features are centered, it becomes the sample covariance matrix.
• → Is a symmetric matrix and positive semidefinite.