Law of large numbers

In probability theory, the weak law of large numbers states that if X₁, X₂, X₃, ... is an infinite sequence of random variables, all of which have the same expected value μ and the same finite variance σ², and they are uncorrelated (i.e., the correlation between any two of them is zero), then the sample average

<math>\overline{X}_n=(X_1+\cdots+X_n)/n</math>

converges in probability to μ. Somewhat less tersely: For any positive number ε, no matter how small, we have

<math>\lim_{n\rightarrow\infty}P\left(\left|\overline{X}_n-\mu\right|<\varepsilon\right)=1.</math>

Chebyshev's inequality is used to prove this result.

A consequence of the weak law of large numbers is the Asymptotic Equipartition Property.

The strong law of large numbers states that if X₁, X₂, X₃, ... is an infinite sequence of random variables that are independent and identically distributed, and have a common expected value μ then

<math>P\left(\lim_{n\rightarrow\infty}\overline{X}_n=\mu\right)=1,</math>

i.e., the sample average converges almost surely to μ.

This law justifies the intuitive interpretation of the expected value of a random variable as the "long-term average when sampling repeatedly".