probability distribution

A probability distribution $p$ is a function that provides the probabilities of occurrence of different possible outcomes in an experiment / of a random variable $X$ at a particular value $x$, given the parameters $\theta$.
In simpler terms, it is a model of the likelihood of an event/outcome/value, given some parameters that characterize a system:

$$X\sim P(X=x\mid \theta )$$

The sum of the probabilities in the distribution must be equal to 1:

$$\sum _{i}P(X=x_i)=1$$

Or for the continuous case:

$${\int }_{-\infty }^{\infty }{f}_X(x)dx=1$$

rework this note after done with measure theory perspective

Wrong:
Probability distributions are classified into two main types:
probability density function (pdf) $f_X(x)$ for continuous variables
probability mass function (pmf) $P(X=x)$ for discrete variables.

Formally:

$$P_X(A)=\{\begin{array}{ll}{\sum }_{x\in A}P(X=x)& \text{if X is discrete}\\ {\int }_A{f}_X(x)dx& \text{if X is continuous}\end{array}\text{for any event }A\subseteq \mathbb{R}$$

So we get the probability density function/probability density function by taking the derivative of the cumulative distribution function:

Some common distributions:
bernoulli distributions model binary outcomes.
binomial distributions are used to model the number of successes in a fixed number of trials.
normal distributions arise in many natural processes; as you add up many random variables, their sum tends to follow a normal distribution, even if they don’t follow a normal distribution individually (Central Limit Theorem).
poisson distributions are used to model the number of events occurring in a fixed interval of time or space; rare events.
exponential distributions are model the probability of waiting times between poisson events.
multinomial distribution is a generalization of the binomial distribution to more than two categories.
geometric distribution
cauchy distribution
pareto distribution
…

machine learning is essentiall learning unknown probability distributions from data.

References

normal dist statquest
article

https://github.com/google-deepmind/distrax/tree/main/distrax/_src/distributions