spectral theorem

Spectral theorem

Every symmetric matrix is diagonalizeable.

---

Let $A\in {\mathbb{R}}^{n\times n}$ be a symmetric matrix. Then, all eigenvalues ${\lambda }_1,\dots ,{\lambda }_n$ of $A$ are real, and there exists an orthonormal basis of ${\mathbb{R}}^n$ consisting of eigenvectors of $A$ (aka eigenbasis).
Then:

$$A=A^T⟹A=Q\Lambda Q^T$$

$Q$ … orthogonal matrix ¹² ($Q^{-1}=Q^T$) whose columns are the eigenvectors of $A$
$\Lambda =\text{diag}({\lambda }_1,\dots ,{\lambda }_n)$
→ For symmetric matrices, this is a pure scaling in the orthogonal directions of the eigenvectors or principal axes - the natural directions along which $A$ acts purely by scaling. Each eigenvector direction $v_k$ is scaled by its eigenvalue ${\lambda }_k$ without any rotation or shearing.
→ Eigendecomposition and SVD coincide for symmetric matrices.

Derivation from the property of eigenvalues

$$\begin{array}{rl}Av& =\lambda v\\ AQ& =Q\Lambda \\ A& =Q\Lambda Q^{-1}\end{array}$$

Footnotes

1. It doesn’t matter if $Q$ is normalized, $Q^{-1}$ cancels the magnitude. ↩

2. For symmetric matrices, the eigenvectors are orthogonal, and if normalized, $Q$ becomes orthogonal ($Q^{-1}=Q^T$). However, for general diagonalizable matrices, eigenvectors need only be linearly independent. ↩