moment (mathematics)

The moments of a function are certain quantitative measures related to the shape of the function’s graph.

The $k^{\text{th}}$ moment of a random variable $X$ is the expected value of the $k^{\text{th}}$ power of $X$:

$$m_k:=\mathbb{E}(X^k)$$

It’s like the taylor series in a way. Under certain conditions, the moments uniquely identify a probability distribution (the “moment problem”). More complex distributions may require more moments to fully describe them.

For a normal distribution, just two parameters fully specify it uniquely: the first moment (mean) and the variance (second central moment). Higher moments exist but are determined by these two.

moment-generating function

The derivative of the moment evaluated at $t=0$ gives the $n^{\text{th}}$ moment of a random variable $X$.

$$m^{(n)}(0)=\mathbb{E}[X^n]$$

Proof

$$\begin{array}{rl}m(t)& ={\int }_{-\infty }^{\infty }{e}^{tx}f(x)dx\\ \left(\frac{dm}{dt}\right)m^{\prime }(t)& ={\int }_{-\infty }^{\infty }x{e}^{tx}f(x)dx⟹m^{\prime }(0)={\int }_{-\infty }^{\infty }xf(x)dx=\mathbb{E}[X]\\ m^{\prime \prime }(t)& ={\int }_{-\infty }^{\infty }{x}^2{e}^{tx}f(x)dx⟹m^{\prime \prime }(0)={\int }_{-\infty }^{\infty }{x}^{2}f(x)dx=\mathbb{E}[X^2]\\ & ⋮\\ m^{(n)}(t)& ={\int }_{-\infty }^{\infty }{x}^n{e}^{tx}f(x)dx⟹m^{(n)}(0)={\int }_{-\infty }^{\infty }{x}^{n}f(x)dx=\mathbb{E}[X^n]\end{array}$$

Moment Generating Function

$$m(t)=\{\begin{array}{ll}{\sum }_x{e}^{tx}P(X=x)& \text{if }X\text{ is discrete}\\ \mathbb{E}[e^{tX}]={\int }_{-\infty }^{\infty }{e}^{tx}f(x)dx& \text{if }X\text{ is continuous}\end{array}$$

The moment generating function uniquely determines the cumulative distribution function $P(X\le x)$.

${\int }_{-\infty }^{\infty }{e}^{tx}f(x)dx$ is the laplace transform of the probability density function $f(x)$.

Moment generating function of the poisson distribution

$$\begin{array}{rl}m(t)& =\sum _{k=0}^{\infty }{e}^{tk}\frac{{\lambda }^k{e}^{-\lambda }}{k!}\\ & =e^{-\lambda }\sum _{k=0}^{\infty }\frac{(\lambda e^t{)}^k}{k!}\\ & =e^{-\lambda }{e}^{\lambda e^t}\\ & =e^{\lambda (e^t-1)}\end{array}$$

Line 1: Plug in formula for poisson distribution
Line 2: Pull $e^{-\lambda }$ from the denominator out of the sum, factor out ${}^k$
Line 3: Recognize the Taylor series expansion of $e^x={\sum }_{k=0}^{\infty }\frac{x^k}{k!}$ with $x=\lambda e^t$
Line 4: Simplify exponent

---

$$\begin{array}{rlrl}{m}^{\prime }(t)& =\lambda e^t{e}^{\lambda (e^t-1)}& & ⟹m^{\prime }(0)=\lambda \\ m^{\prime \prime }(t)& =(\lambda e^t+{\lambda }^2{e}^{2t})e^{\lambda (e^t-1)}& & ⟹m^{\prime \prime }(0)=\lambda +{\lambda }^2\end{array}$$

$\text{Var}(X)=m^{\prime \prime }(0)-(m^{\prime }(0){)}^2=\lambda +{\lambda }^2-{\lambda }^2=\lambda$

Convolution property of MGFs $X,Y$ be independent random variables and $m_X,m_Y$ their MGFs, then define $Z=X+Y⟹m_Z^{(t)}=m_X(t)\cdot m_Y(t)$. Generally, for $S_n={\sum }_i^n{X}_i$

Let

$$m_s^{(t)}=\prod _i^n{m}_{X_i}(t)$$

https://andrewcharlesjones.github.io/journal/moment-generating-functions.html

Link to original

---

https://de.wikipedia.org/wiki/Moment_(Stochastik)