logarithm

Logarithm

For any positive numbers $M,N$ and positive base $b\ne 1$:
A logarithm tells us what exponent we need to raise the base to get a number:
If $b^y=x$ then ${\log }_b(x)=y$
Written as ${\log }_b(x)$ or $\ln (x)$ for natural log (base $e$)
The domain of ${\log }_b(x)$ is $x\in {\mathbb{R}}^{+}$
Higher base $b$ means more “compressed” representation (smaller log values for same $x$)

Basic Properties

${\log }_b(M\cdot N)={\log }_b(M)+{\log }_b(N)$
${\log }_b(\frac{M}{N})={\log }_b(M)-{\log }_b(N)$
${\log }_b(M^n)=n\cdot {\log }_b(M)$
${\log }_b(b)=1$
${\log }_b(1)=0$
${\log }_b(b^n)=n$
${\log }_b(x)=\frac{\ln (x)}{\ln (b)}$ (change of basis)
$b^{{\log }_b(x)}=x$

Logarithm turns multiplication into addition

The logarithm is an isomorphism between $(R^{+},\cdot )$ and $(R,+)$:

$$\log :(R^{+},\cdot )\to (R,+)$$

It transforms multiplication into addition: $\log (xy)=\log (x)+\log (y)$

This satisfies all isomorphism requirements:

• Homomorphism: Preserves group operation (multiplication → addition)
• Bijective: One-to-one correspondence via $\exp$ as inverse
• Identity preservation: $\log (1)=0$
• Inverse preservation: $\log (1/x)=-\log (x)$

Why we want additive instead of multiplicative

Linear algebra works:
Once in log-space, we can use all the tools of linear algebra. Exponential relationships $y=e^b\cdot e^{mx}$ become linear $\log y=mx+b$, so we can use linear regression to find best-fit parameters, generally, matrix operations work on log-transformed data.

Averaging becomes meaningful: Arithmetic mean of log-values $\frac{1}{n}\sum \log x_i=\log (\sqrt[n]{\prod x_i})$ gives the geometric mean - the “typical” multiplicative factor. For growth rates: average $2\times$ and $8\times$ growth isn’t $5\times$ (arithmetic) but $4\times$ (geometric: $\sqrt{2\cdot 8}$).

Domain expansion: Multiplicative operations are trapped in $(0,\infty )$ - can’t represent “negative growth” or pass through zero. Log maps this to $(-\infty ,\infty )$ where we can freely add/subtract without worrying about sign constraints or division by zero.

Numerical stability: Multiplying probabilities like $0.01^{100}$ gives $10^{-200}$ (underflows to 0 in float64). In log-space: $100\cdot \log (0.01)=-200$, perfectly representable. Similarly, $1000000^{100}$ overflows, but $100\cdot \log (10^6)=600$ doesn’t.

Probability & Information:
Independent events have probabilities that multiply: $P(A\cap B)=P(A)\cdot P(B)$
In log-space they add: $\log P(A\cap B)=\log P(A)+\log P(B)$
→ We can sum evidence from multiple sources. Log-likelihood ratios add up. Maximum likelihood becomes a sum to maximize, not a product.
This additivity defines information: $I(x)=-\log P(x)$ bits. Rare events (small $P$) have large positive information.
Entropy $H=-\sum p_i\log p_i$ becomes extensive: entropy of two independent systems equals sum of individual entropies.

Signal processing:
Each amplifier/attenuator multiplies signal: ${\text{signal}}_{\text{out}}=G_1\cdot G_2\cdot G_3\cdot {\text{signal}}_{\text{in}}$
In decibels they add: $20{\log }_{10}(G_1)+20{\log }_{10}(G_2)+20{\log }_{10}(G_3)$ dB
→ Can quickly calculate total system gain by adding component gains. A $1000\times$ amplifier followed by $0.01\times$ attenuator? That’s 60 dB + (-40 dB) = 20 dB = $10\times$ total gain.
Also compresses huge dynamic range: microvolts to kilovolts ($10^9$ range) becomes manageable -180 to +60 dBV.

Chemistry: Hydrogen ion concentration spans $10^{-14}$ to $10^0$ mol/L (100 trillion-fold range!)
pH scale makes this linear: pH 0-14, where each unit = $10\times$ concentration change
Reaction energies add because $\Delta G=-RT\ln K$ converts multiplicative equilibrium constant $K$ to additive free energy

Growth processes: Exponential growth $x(t)=x_0{e}^{rt}$ becomes linear in log: $\log x(t)=\log x_0+rt$
Now it’s a straight line with slope $r$ - can estimate growth rate from any two points
Can use linear regression to fit exponential models, detect deviations from exponential growth

Log isn't just more compact notation, but literally saves memory and compute if we have numbers that have lots of zeros.