A latent variable is a hidden trait the model assumes exists but you can’t measure directly—only infer from other things you can observe.
Examples:
- “Math ability” inferred from test answers.
- “Depression severity” inferred from survey items and behavior.
- “Topics” in documents inferred from word patterns
Why use them?
- They explain correlations between observed variables.
- They reduce dimensionality (factor analysis, PCA).
- They let models capture structure we can’t measure directly.
Two views of latent variables
In a probabilistic sense, the “classic latent variable” is an unobserved random variable in a generative model that explains the data.
In a representation learning/geometric sense, latent variables define coordinates in a lower-dimensional latent space where data points are embedded. A learned code that isn’t directly observed but captures essential features of the data. It’s deterministic, given , unless you introduce randomness on top.
Variational Autoencoders combine both views: They explicitly model latent variables as random variables, while also learning a latent space representation.