Zipf's law

Zipf's law

The $k$-th most common item in a ranked distribution has frequency $\propto 1/k^{\alpha }$, where $\alpha \approx 1$.

$$f(k)\propto \frac{1}{k^{\alpha }}$$

$k$ … rank (1st most common, 2nd most common, …)
$\alpha$ … exponent, empirically close to 1 in many domains

A power law over discrete ranked items. The “law” is the empirical observation that $\alpha \approx 1$ keeps showing up across unrelated systems.

The distribution was originally observed in word frequencies, but Zipf’s broader claim (human-behavior-and-the-principle-of-least-effort) was that it reflects a general principle of efficiency: systems under optimization pressure converge on this distribution because it balances the cost of maintaining many distinct items against the cost of reusing too few.

EXAMPLE

Word frequencies: “the” is ~7% of English text, “of” ~3.5%, “and” ~2.8%, … the 1000th most common word is ~1000x rarer than the most common.

Word meanings: the number of distinct meanings of a word scales as the square root of its frequency. Frequent words can absorb many meanings cheaply because they show up in rich, disambiguating contexts!

Cities: the largest city in a country tends to be ~2x the second largest, ~3x the third, …

Wealth: income distributions follow similar rank-frequency patterns.

Web traffic: a few sites get most visits, with a long tail of small sites.

Biology: species abundance in ecosystems, gene expression levels.