joint distribution

Joint distribution 🌿

For two random variables $X,Y$, their joint probability distribution is a probability distribution that gives the probability of each possible pair of outcomes $(x,y)$ for $X$ and $Y$. It is denoted as $P(X=x,Y=y)$ or simply $P(x,y)$ for discrete variables, and $f(x,y)$ for the density in continuous cases.

Notation

$P(x,y)$: Probability mass function (discrete) - gives actual probability values
$f(x,y)$: Probability density function (continuous) - must be integrated to get probabilities
$f_X(x)$: Marginal density of $X$
$f_{X|Y}(x|y)$: Conditional density of $X$ given $Y=y$

Probability from density

The probability of $(x,y)$ being in some 2D area $A$ is the integral of the PDF $f(x,y)$ over that area:

$$\begin{array}{r}P((x,y)\in A)={\int }_{A}f(x,y)dxdy\end{array}$$

Marginal density

The marginal density of a variable is obtained by integrating the joint density over the other variable:

$$\begin{array}{rl}{f}_X(x)& ={\int }_{-\infty }^{\infty }f(x,y)dy\end{array}$$

Here we’re integrating out (averaging over all possible values of) $Y$, to get the marginal density of $X$, i.e. the density of $X$ regardless of $Y$.
This also work with discrete distributions: $P(X=x)={\sum }_{i}P(X=x,Y=y_i)$

Conditional probability

The conditional probability of $X$ given $Y=y$ is given by:

$$\begin{array}{rl}P(X=x\mid Y=y)& =\frac{P(x,y)}{P_Y(y)}\end{array}$$

This is almost identical to the standard conditional probability (probability of $X$ AND $Y$ divided by the probability of $Y$), but more general, as it’s a conditional density function of all values of $x$ and $y$.
This is the same formula but using density notation - $f_{X|Y}(x|y)$ denotes the conditional density function of $X$ given $Y=y$:

$$f_{X|Y}(x|y)=\frac{f(x,y)}{f_Y(y)}$$