eigenvalue

Eigenvectors are vectors who’s span is unchanged by a linear transformation.

Eigenvectors only get stretched/squished (scaled) or are left unchanged.
The eigenvalue $\lambda$ is the value by which the vector is scaled.
You can think of it as the axis of rotation.
For a true rotation however, the eigenvalue needs to be $\pm 1$, as rotations don’t scale vectors (we call those transformation matrices unitary)

Applying the transformation matrix $A$ to the eigenvector $\vec{v}$ is equivalent to simply scaling it by its eigenvalue $\lambda$:

$$A\vec{v}=\lambda \vec{v}$$

We want to figure out the eigenvector $\vec{v}$ and the eigenvalue $\lambda$:
For that we make both sides look like matrix-vector multiplication, by multiplying $\lambda$ with the identity.

$$A\vec{v}=(\lambda I)\vec{v}$$

We now want the nonzero solution for $\vec{v}$:

$$\begin{array}{rl}(A-\lambda I)\vec{v}& =\vec{0}\end{array}$$

The product of a nonzero vector with a nonzero matrix can only be zero if the determinant is zero, so we are looking for a $\lambda$ that satisfies: $\det (A-\lambda I)=0$
Meaning we tweak lambda in such a way, that the resulting transformation squishes space to a lower dimension:

If $\det (A\vec{v})=0$, then the transformation changes the space in a way that it collapses to a lower dimension (for the case of ${\mathbb{R}}^3$ it may be a plane, line or point: anything with 0 volume).


If the determinant is zero, the matrix is not invertible.

Link to original

We can now find the eigenvector $\vec{v}$, which stays on its span, scaled by $\lambda$, by solving the equation $(A-\lambda I)\vec{v}=\vec{0}$.

Not every transformation has a real eigenvector.

A $90°$ rotation e.g.

$$\begin{array}{rl}\det \left(\left[\begin{array}{cc}-\lambda & -1\\ 1& -\lambda \end{array}\right]\right)& =(-\lambda \cdot -\lambda )-(-1\cdot 1)={\lambda }^2+1=0\\ & \to \lambda =i\text{ or }\lambda =-i\end{array}$$

only has the imaginary eigenvectors $i$ and $-i$:
The axis of rotation is around the complex plane, at the origin.
So multiplying with $i$ usually means some kind of rotation. This is also the case within the complex plane (the angles get added together).

A shear like $\left[\begin{array}{cc}1& 1\\ 0& 1\end{array}\right]$ only has a single eigenvalue.

$$\begin{array}{rl}& \det \left(\left[\begin{array}{cc}1-\lambda & 1\\ 0& 1-\lambda \end{array}\right]\right)=(1-\lambda )\cdot (1-\lambda )=0\\ & \to \lambda =1\end{array}$$

The eigenspace is 1-dimensional … a single eigenline:

In a diagonal matrix, all vectors are eigenvectors, and the diagonal entries are the eigenvalues.

Diagonal matrices only scale what they are being multiplied with, since the basis vectors are also eigenvectors. For diagonal matrices, every vector is an eigenvector.

The matrix $\left[\begin{array}{cc}2& 0\\ 0& 2\end{array}\right]$ scales every vector by 2. Every vector is an eigenvector with $\lambda =2$.

Diagonal matrices have nice computational properties when taking large powers.

$${\left[\begin{array}{cc}a& 0\\ 0& b\end{array}\right]}^k\cdot \left[\begin{array}{c}x\\ y\end{array}\right]=\left[\begin{array}{cc}{a}^{100}& 0\\ 0& b^{100}\end{array}\right]\cdot \left[\begin{array}{c}x\\ y\end{array}\right]$$

Link to original

Diagonalization: Performing a transformation from the perspective of the eigenbasis.

We can exploit the efficiency of diagonal matrices for taking large powers if we convert a transformation matrix $A$ into a diagonal one by multiplying it with the change-of-basis matrix $S$ and its inverse:

$$D=S^{-1}AS$$

“First translate from the language of the eigenbasis to our basis, then apply $A$, then translate back. The resulting matrix $D$ will represent the same transformation as $A$, just in the language of the eigenbasis.”

If $S$ consists of the eigenvectors of $A$, we are guaranteed to have a diagonal matrix with eigenvalues down the diagonal. → This is because we are then working in a coordinate system where the basis vectors get scaled by the transformation!

Then we can compute $A^n$ like so:

$$A^n=S(D^n)S^{-1}$$

Not every $A$ has enough eigenvectors to form proper eigenspace — the eigenspace needs to have the same dimension as the original space (rank $n$) – so this trick is not always possible, like for a shear.
Symmetric matrices are always diagonalizable.

Link to original

Eigenvector puzzle

3b1b’s solution: Eigenvalue puzzle solution.pdf

Highlights from the solution / my take:

$$\begin{array}{rl}A& =\left[\begin{array}{cc}0& 1\\ 1& 1\end{array}\right]\\ A^2& =\left[\begin{array}{cc}1& 1\\ 1& 2\end{array}\right]\\ A^3& =\left[\begin{array}{cc}1& 2\\ 2& 3\end{array}\right]\\ A^4& =\left[\begin{array}{cc}2& 5\\ 5& 8\end{array}\right]\\ A^5& =\left[\begin{array}{cc}5& 7\\ 7& 13\end{array}\right]\end{array}$$

The terms form the fibonacci sequence:

$$\left[\begin{array}{cc}0& 1\\ 1& 1\end{array}\right]\left[\begin{array}{c}a\\ b\end{array}\right]=\left[\begin{array}{c}b\\ a+b\end{array}\right]$$

The eigenvalues of $A$ were computed by:

$$\begin{array}{rl}& \det ([A-\lambda I])=\det \left(\left[\begin{array}{cc}0-\lambda & 1\\ 1& 1-\lambda \end{array}\right]\right)={\lambda }^2-\lambda -1=0\\ & \to {\lambda }_{1,2}=\frac{1\pm \sqrt{5}}{2}\end{array}$$

… ${\lambda }_1$ is the golden ratio $\varphi =1.618\dots$, and ${\lambda }_2$ is $-\frac{1}{\varphi }=-0.618\dots$.

(after calculating the eigenvectors …)

Converting to the eigenbasis:

$$\begin{array}{rl}D& =S^{-1}AS\\ & ={\left[\begin{array}{cc}2& 2\\ 1+\sqrt{5}& 1-\sqrt{5}\end{array}\right]}^{-1}\left[\begin{array}{cc}0& 1\\ 1& 1\end{array}\right]\left[\begin{array}{cc}2& 2\\ 1+\sqrt{5}& 1-\sqrt{5}\end{array}\right]\\ & ⋮\\ & =\left[\begin{array}{cc}{\lambda }_1& 0\\ 0& {\lambda }_2\end{array}\right]\end{array}$$

Now we can compute:

$$D^n=\left[\begin{array}{cc}{\lambda }_1^n& 0\\ 0& {\lambda }_2^n\end{array}\right]$$

…instead of tedious matrix multiplication.

To translate back into the original cordinate system: $A^n=S(D^n)S^{-1}$

Okay I’m not typing this shit, looks sick tho (look at the solution pdf for the equation).
It’s a really interesting / funny way of computing fibonacci.
(And ig probably the most efficient? As you don’t need any recursion)

References

vector

3b1b: Eigenvectors and eigenvalues | Chapter 14, Essence of linear algebra

https://www.khanacademy.org/math/linear-algebra/alternate-bases/eigen-everything/v/linear-algebra-introduction-to-eigenvalues-and-eigenvectors