pseudo inverse

Moore-penrose pseudo inverse

The Moore-Penrose pseudo-inverse $A^{+}$ is a generalization of the inverse matrix to non-square matrices $A\in {\mathbb{R}}^{m\times n}$. It is defined as:

$$\begin{array}{r}A=U\Sigma V^T⟹A^{+}=V{\Sigma }^{-1}{U}^T\end{array}$$ $$\begin{array}{rl}Ax& =b\\ \underset{\text{SVD}}{\underbrace{U\Sigma V^T}}x& =b\\ \text{cancelto}IV{\Sigma }^{-1}{U}^{T}U\Sigma V^T\tilde{x}& =\underset{:=A^{+}}{\underbrace{V{\Sigma }^{-1}{U}^T}}b\end{array}$$

Here we used the SVD of $A$ to find the left pseudo inverse of $A$, or rather an approximation (hence $\tilde{x}$).
This is the best $x$ we can find which minimizes the error $||Ax-b|{|}_2$ – the minimum norm solution:

$$\begin{array}{r}\min \Vert \tilde{x}{\Vert }_2\text{s.t.}A\tilde{x}=b\end{array}$$

This minimum norm criterion is particularly relevant for underdetermined systems, where we have infinitely many exact solutions and want to select the one requiring least energy. The $L_2$ norm directly corresponds to energy in many physical systems (e.g., electrical, mechanical).

For overdetermined systems, the pseudo-inverse instead gives us the least squares solution, minimizing $\Vert Ax-b{\Vert }_2$ when no exact solution exists.

Properties

The Moore-Penrose pseudo-inverse satisfies four conditions:
$A{A}^{+}A=A$
$A^{+}A{A}^{+}=A^{+}$ (weak inverse)
$(A{A}^{+}{)}^T=A{A}^{+}$
$(A^{+}A{)}^T=A^{+}A$