factorization

Info

Technically, any product form is technically a “factorization”, but I/we usually reserve the term for structure-based simplifications, described below.

Factorization of multivariate distributions

If you have a multivariate distribution with independent underlying distributions, then you can factorize these underlying distributions.

$$p(x_1,x_2,\dots ,x_n)=p(x_1)p(x_2)\dots p(x_n)$$

E.g. how much a certain car color is liked and how much a certain car model is liked: $p(\text{model, color})=p(\text{model})\cdot p(\text{color})$

This is a special case of the chain rule of probability.

It is enough to know the two factors seperately to predict the joint distribution!

This allows to predict values for combinations which you have not encountered before, you also need to store much less information this way: $O(n)$ instead of $O(2^n)$.

Factorization in the brain

The brain applies factorization in many areas, one example is hippocampal remapping.
Factorizing environments (or all kinds of concepts - in the neocortex) into building blocks allows to easily build new models on the fly.

References

How Your Brain Organizes Information - Can We Build an Artificial Hippocampus - Artem Kirsanov