bayesian inference

Bayesian Inference

Treats all parameters ¹ as random variables with a prior distribution $p(\theta )$.
Update via Bayes’ rule:

$$p(\theta \mid X)=\frac{p(X\mid \theta )p(\theta )}{p(X)}$$

The posterior $p(\theta \mid X)$ represents our updated beliefs about parameters after seeing data.

Computing it often requires intractable integrals (for the marginal likelihood $p(X)$), leading to approximation methods:

• Variational inference - approximate posterior with simpler distribution
• Markov chain Monte Carlo - sample from posterior distribution
• Laplace approximation - Gaussian approximation around posterior mode

bayesianism
bayesian neural network

Footnotes

1. Here “parameters” includes all unobserved quantities, not just model parameters, but also latent variables, or hyperparameters. In practice, some may be kept fixed/point-estimated rather than given priors, depending on the model and computational considerations. ↩