matrix factorization

matrix factorization

A matrix factorization/decomposition is an exact multiplicative representation of a matrix as a product of matrices.

$$A=F_1{F}_2\cdots F_k$$

where $F_i$ are matrices which usually have special structure (e.g. triangular matrix, orthogonal matrix, diagonal matrix, etc.) that make them easier to work with/simplify computation.

Examples: LU, QR, Cholesky, SVD, Schur, etc.

A low-rank approximation of a matrix via learned matrix factorization.

Given $X\in {\mathbb{R}}^{m\times n}$, choose rank $k\ll \min (m,n)$ and learn

$$X\approx U{V}^{\top },U\in {\mathbb{R}}^{m\times k},V\in {\mathbb{R}}^{n\times k}.$$

Typically fit by minimizing a regularized loss over observed entries $\Omega$:

$$\underset{U,V}{\min }\sum _{(i,j)\in \Omega }(X_{ij}-U_i^{\top }{V}_j{)}^2+\lambda (\Vert U{\Vert }_F^2+\Vert V{\Vert }_F^2).$$

Unlike plain SVD, MF handles sparsity/missing data and adds regularization.

This can be used nicely for recommender systems.