probability density function

Rewrite this note

A PDF is a probability distribution that describes a continuous random variable taking on values in different ranges.
The density value represents “probability per unit length/area/volume”, not probability itself, so it can be greater than 1 (e.g., $f(x)=2$ for $x\in \mathcal{U}(0,0.5)$)

But the total area/volume under the curve must equal 1: ${\int }_{-\infty }^{\infty }f(x)dx=1$
You can imagine poking the curve like a waterballloon… if it goes down somewhere, it must go up somewhere else to compensate.

It’s a model of a random process:
→ You can do hypothesis testing with data
→ You can estimate parameters (mean, std, …) with a small sample

$f:{\mathbb{R}}^d\to [0,\infty )$
$P(X\in A)={\int }_{A}f(x)dx$ for any measurable set $A\subseteq {\mathbb{R}}^d$
“densitiy w.r.t. base measure $\mu$ ⇒ $\mu$ = counting measure → pmf, $\mu$ = lebesgue measure → pdf”

Density

$f(x_0)$ is not a probability but the rate of accumulation of probability near $x_0$: $f(x_0)=F^{\prime }(x_0)$ the derivative of the CDF at that point.
We call it density because it describes how tightly probability is packed around that point.
The relative density between two points $x_1$ and $x_2$ describes how much more likely it is to find a random sample near $x_1$ than near $x_2$ → We get

If $X\sim \text{Uniform}(0,0.1)$, what is $f(0.05)$?

The PDF of the uniform distribution is constant $c$ on $[0,0.1]$ and must integrate to $1$::

$${\int }_0^{0.1}cdx=c\cdot (0.1-0)=0.1c=1⟹c=10$$

→ $f(0.05)=10$.

The probability of a random variable falling within a range is the integral of the PDF over that range:

$$P(a\le x\le b)={\int }_a^{b}f(x)dx$$

→ Probabilities are only meaningful over intervals/ranges, not individual points.
→ The value of $f(x)$ can be greater than $1$, as it represents density not probability.
→ $P(X=x)=0$ for any single point $x$, as it represents an interval of width zero.

Example: For a standard normal distribution, $f(0)\approx 0.399$, but $P(X=0)=0$, however, $P(-1\le X\le 1)\approx 0.683$, representing the probability of falling within that interval.

So while taking sums of probabilities over individual points works in a discrete context (probability mass function):

$$P(x\in \{a,b,c\})=P(x=a)+P(x=b)+P(x=c)=\sum _{x\in \{a,b,c\}}P(x)$$

It doesn’t work in a continuous context:

$$\begin{array}{r}P(x\in [a,c])\cancel{=\sum _{x\in [a,c]}P(x)}={\int }_{x\in [a,c]}P(x)dx\end{array}$$

If $X\sim \text{Uniform}(0,2)$, what is $P(0.95\le X\le 1.05)$?

The PDF of the uniform distribution is constant $c$ on $[0,2]$ and must integrate to $1$:

$${\int }_0^{2}cdx=c\cdot (2-0)=2c=1⟹c=0.5$$

→ $f(x)=0.5$ for $x\in [0,2]$.
So the probability of $X$ falling within $[0.95,1.05]$ is:

$$P(0.95\le X\le 1.05)={\int }_{0.95}^{1.05}0.5dx=0.5\cdot (1.05-0.95)=0.5\cdot 0.1=0.05$$

That’s the area of the rectangle formed by the interval (width 0.1) and the density (height 0.5).