exponential distribution

The exponential distribution is a continuous probability distribution often used to model the time elapsed between events. A key characteristic is its memoryless property.

Applications

The exponential distribution is useful for modeling various real-world phenomena, such as:

• Lifetimes of light bulbs or electronic components.
• Time until radioactive decay of an element.
• Service times, for example a call center.

Exponentially distributed random variable

A continuous random variable $T$ follows an exponential distribution with a rate parameter $\lambda >0$, denoted as $T\sim \text{Exponential}(\lambda )$.

Its Probability Density Function (PDF) is given by:

$$P(T=t)=p(t)=\{\begin{array}{ll}\lambda e^{-\lambda t}& \text{for }t\ge 0\\ 0& \text{for }t<0\end{array}$$

Here, $\lambda$ is the rate of the process $\lambda =1/\mathbb{E}[T]$, where $\mathbb{E}[T]$ is the expected value of $T$.
So the mean of the distribution is $1/\lambda$, the variance is $1/{\lambda }^2$, and the standard deviation is $1/\lambda$ too.

Memoryless Property

The memoryless property means that the probability of an event occurring in the future is independent of how much time has already passed.
Mathematically, for $dt,t\ge 0$:

$$P(T>t+dt\mid T>dt)=P(T>t)$$

“If $dt$ amount of time has already passed, the probability that it last another $t$ amount of time after that is the same as the distribution starting from $0$/now again.” (continually resetting probability)
This makes it suitable for modeling components that do not “age” in the sense of becoming more or less likely to fail over time, given they haven’t failed yet.

Hazard Rate

The hazard rate (or failure rate) $h(t)$ represents the instantaneous rate of failure at time $t$, given survival up to that time. For the exponential distribution, it’s defined as:

$$h(t)=\frac{p(t)}{P(T>t)}=\frac{\lambda e^{-\lambda t}}{e^{-\lambda t}}=\lambda$$

The constant hazard rate $\lambda$ is a direct consequence of the memoryless property - the failure rate doesn’t change over time. This means an exponentially distributed component is just as likely to fail in the next instant whether it’s brand new or has been running for years.

The relationship $P(T<t+dt|T>t)=\lambda \cdot dt$ (given I’ve lasted $t$ time, what’s the probability I’ll last another $dt$) tells us that for a small time interval $dt$, the probability of failure is approximately $\lambda$ times the length of that interval, regardless of how long the component has already survived.

\begin{align*}
P(T<t + dt | T > t) &= 1 - P(T> t+dt|T>t) \
&= 1 -P(T>t) \quad \text{memoryless property} \
&= 1-e^{-\lambda dt}
\end{align*}

When we do [[taylor expansion]] for the exponential and make a small $dt$ approximation, we get the hazard rate:

= 1- [1 - \lambda dt + \frac{1}{2}\lambda^{2} dt^{2} - \dots]
\approx \lambda dt

$\implies P(t \lt T \le t + dt) = \lambda P(T \gt t)dt$

---

Exercise 1: Derive the Cumulative Distribution Function (CDF)

The Cumulative Distribution Function $F(t)$ is defined as $F(t)=P(T\le t)={\int }_{-\infty }^{t}p(x)dx$. We want to show that for $t\ge 0$, $F(t)=1-e^{-\lambda t}$.

---

1. Define the PDF $p(x)$:

$$p(x)=\{\begin{array}{ll}\lambda e^{-\lambda x}& \text{for }x\ge 0\\ 0& \text{for }x<0\end{array}$$

2. Consider the case for $t<0$:
   For $t<0$, $p(x)=0$ over the entire integration range $(-\infty ,t]$.

$$F(t)={\int }_{-\infty }^{t}0dx=0$$

3. Consider the case for $t\ge 0$:
   The integral is split based on the definition of $p(x)$:

$$F(t)={\int }_{-\infty }^{t}p(x)dx={\int }_{-\infty }^{0}p(x)dx+{\int }_0^{t}p(x)dx$$

Since $p(x)=0$ for $x<0$:

$$F(t)={\int }_{-\infty }^{0}0dx+{\int }_0^t\lambda e^{-\lambda x}dx=0+\lambda {\int }_0^t{e}^{-\lambda x}dx$$

4. Evaluate the integral:

$$\lambda {\int }_0^t{e}^{-\lambda x}dx=\lambda {\left[\frac{e^{-\lambda x}}{-\lambda }\right]}_0^t={\left[-e^{-\lambda x}\right]}_0^t$$

5. Substitute the limits:

$$F(t)=(-e^{-\lambda t})-(-e^{-\lambda \cdot 0})=-e^{-\lambda t}-(-e^0)$$

Since $e^0=1$:

$$F(t)=-e^{-\lambda t}+1=1-e^{-\lambda t}$$

Exercise 2: Verify that $P(t)$ is a valid Probability Density Function (PDF)

To verify that $p(t)$ (as defined below) is a valid PDF, we need to show two conditions:

1. $p(t)\ge 0$ for all $t$.
2. ${\int }_{-\infty }^{\infty }p(t)dt=1$.

---

Condition 1: $p(t)\ge 0$ for all $t$
The PDF is defined as:

$$p(t)=\{\begin{array}{ll}\lambda e^{-\lambda t}& \text{for }t\ge 0\\ 0& \text{for }t<0\end{array}$$

• For $t<0$, $p(t)=0$, which is $\ge 0$.
• For $t\ge 0$, $p(t)=\lambda e^{-\lambda t}$. Since $\lambda >0$ (rate parameter) and $e^{-\lambda t}$ is always positive for any real $\lambda t$, their product $\lambda e^{-\lambda t}$ is also $\ge 0$.

Thus, $p(t)\ge 0$ for all $t$.

Condition 2: ${\int }_{-\infty }^{\infty }p(t)dt=1$
We need to integrate $p(t)$ over its entire domain:

$${\int }_{-\infty }^{\infty }p(t)dt={\int }_{-\infty }^{0}p(t)dt+{\int }_0^{\infty }p(t)dt$$

Since $p(t)=0$ for $t<0$:

$${\int }_{-\infty }^{\infty }p(t)dt={\int }_{-\infty }^{0}0dt+{\int }_0^{\infty }\lambda e^{-\lambda t}dt=0+\lambda {\int }_0^{\infty }{e}^{-\lambda t}dt$$

Now, evaluate the definite integral:

$$\lambda {\int }_0^{\infty }{e}^{-\lambda t}dt=\lambda {\left[\frac{e^{-\lambda t}}{-\lambda }\right]}_0^{\infty }={\left[-e^{-\lambda t}\right]}_0^{\infty }$$

This involves a limit:

$$=\underset{b\to \infty }{\lim }[-e^{-\lambda t}{]}_0^b=\underset{b\to \infty }{\lim }(-e^{-\lambda b})-(-e^{-\lambda \cdot 0})$$

Since $\lambda >0$, as $b\to \infty$, $-\lambda b\to -\infty$, so $e^{-\lambda b}\to 0$. And $e^0=1$.

$$=(0)-(-1)=1$$

Both conditions are satisfied, so $p(t)$ is a valid PDF.