independent

Independent events

Two events $A$ and $B$ are independent if knowing $A$ gives no extra information about $B$, and vice versa:

$$\begin{array}{r}P(A\mid B)=P(A)\\ P(B\mid A)=P(B)\end{array}$$

Equivalently, using the formula of conditional probability, for independent events, we can say:

$$\begin{array}{rl}P(A)=P(A\mid B)& =\frac{P(A\cap B)}{P(B)}\\ ⟹P(A\cap B)& =P(A)\cdot P(B)\end{array}$$

So the chain rule of probability simplifies to multiplication for independent events.

EXAMPLE

$\Omega$ = deck of 52 cards
$A$ = card is spade → $P(A)=\frac{1}{4}$
$B$ = card is queen → $P(B)=\frac{1}{13}$
$P(A\cap B)=P(A)P(B)=\frac{1}{52}$
The probability of getting a spade or queen doesn’t change if we restrict ourselves to the set $A$ or $B$:

Independence is about proportions, not overlap.

$A$ and $B$ are independent exactly when $A$ takes up the same fraction of $B$ as it does of all of $\Omega$, and vice versa.

$A$ is a vertical strip of width $P(A)$, $B$ a horizontal strip of height $P(B)$, and their lens is the rectangle with area $=\text{width}\times \text{height}=P(A)P(B)$. Inside band $B$, $A$ still occupies the same width $P(A)$: knowing $B$ happened tells you nothing about $A$.

Disjoint is the opposite of independent

Two disjoint events with positive probability are maximally dependent: if $A$ happens, $B$ definitely didn’t; $P(A\mid B)=0\ne P(A)$.

Independent random variables

Two random variables are independent:

$$X\perp Y⟺p(X,Y)=p(X)p(Y)$$

Knowing one variable gives no information about the other.

Mutual independence (more than 2 variables):

$$p(X_1,X_2,\dots ,X_n)=\prod _{i=1}^{n}p(X_i)$$

What does it mean for two events to be independent?

Neither gives you any information about the other.

$$\begin{array}{rl}P(A\mid B)& =P(A)\\ P(B\mid A)& =P(B)\\ P(A\cap B)& =P(A)\cdot P(B)\end{array}$$

$A$ and $B$ are independent exactly when $A$ takes up the same fraction of $B$ as it does of all of $\Omega$, and vice versa.