v1 · padrão canônico

Lesson 77 — Central Limit Theorem

The mean of n iid random variables converges to a normal distribution regardless of the original distribution — the most important law in statistics. Proof via characteristic function, Berry-Esseen bound, and inference applications.

Used in: 2.º ano do EM (16-17 anos) · Math B japonês §4.4 · Stochastik LK alemão · H2 Math singapurense cap. 21

\bar X_n \xrightarrow{d} \mathcal{N}\!\left(\mu,\,\frac{\sigma^2}{n}\right) \quad (n \to \infty)

Choose your door

Rigorous notation, full derivation, hypotheses

Formal statement and proof

Lindeberg-Lévy version

Definition· Central Limit Theorem (classical version)

Let $X_1, X_2, \ldots$ be iid (independent and identically distributed) random variables with $E[X_i] = \mu \in \mathbb{R}$ and $\text{Var}(X_i) = \sigma^2 \in (0, +\infty)$ . Define the standardized statistic

$Z_n = \frac{\bar X_n - \mu}{\sigma/\sqrt{n}}, \quad \bar X_n = \frac{1}{n}\sum_{i=1}^n X_i.$

Then $Z_n \xrightarrow{d} \mathcal{N}(0, 1)$ : for every $z \in \mathbb{R}$ ,

$\lim_{n \to \infty} P(Z_n \leq z) = \Phi(z) = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^z e^{-t^2/2}\,dt.$

"The central limit theorem is the unofficial sovereign of probability theory." — Grinstead & Snell, Introduction to Probability, §9.1

Version for sums

If $S_n = X_1 + \cdots + X_n$ , then $S_n \approx \mathcal{N}(n\mu,\, n\sigma^2)$ for large $n$ .

Z_n = \frac{S_n - n\mu}{\sigma\sqrt{n}} \xrightarrow{d} \mathcal{N}(0,1)

what this means · Standardization of the sum: same formula, different scale.

Convergence speed: Berry-Esseen inequality

Proof sketch via characteristic function

Let $Y_i = (X_i - \mu)/\sigma$ (zero mean, unit variance). Taylor expansion of $\varphi_{Y_i}$ :

$\varphi_{Y_i}(t) = 1 - \frac{t^2}{2} + o(t^2) \quad (t \to 0).$

For $Z_n = (Y_1 + \cdots + Y_n)/\sqrt{n}$ :

$\varphi_{Z_n}(t) = \left[\varphi_{Y_i}\!\left(\frac{t}{\sqrt{n}}\right)\right]^n = \left[1 - \frac{t^2}{2n} + o\!\left(\frac{1}{n}\right)\right]^n \xrightarrow{n\to\infty} e^{-t^2/2}.$

But $e^{-t^2/2}$ is the characteristic function of $\mathcal{N}(0, 1)$ . The Lévy's Continuity Theorem concludes $Z_n \xrightarrow{d} \mathcal{N}(0,1)$ . $\blacksquare$

When the CLT does not hold

Essential hypotheses

Independence (minimum sufficient; relaxable to $\alpha$ -mixing).
Finite variance $\sigma^2 < \infty$ .
n sufficiently large — rule of thumb: $n \geq 30$ for not-too-skewed distributions; $n \geq 100$ for high skewness.

Solved examples

Example— 1· Average weight of packages posted at the Post Office

Problem. Packages posted at the post office have a mean weight $\mu = 2{,}4$ kg and standard deviation $\sigma = 0{,}8$ kg (unknown distribution). A screening samples 64 packages. What is the probability that the sample mean weight exceeds 2.6 kg?

Strategy. By CLT, $\bar X_{64} \approx \mathcal{N}(2{,}4,\; 0{,}8^2/64)$ . Standardize and consult z-table.

Resolution.

$\sigma_{\bar X} = 0{,}8/\sqrt{64} = 0{,}8/8 = 0{,}1$ kg.

$z = \frac{2{,}6 - 2{,}4}{0{,}1} = \frac{0{,}2}{0{,}1} = 2{,}0.$

$P(\bar X > 2{,}6) = P(Z > 2{,}0) = 1 - \Phi(2{,}0) = 1 - 0{,}9772 = 0{,}0228 \approx 2{,}3\%$ .

Verification. 68-95-99.7 rule: $\pm 2\sigma_{\bar X}$ contains 95%, so 2.5% in each tail. Result of 2.3% is consistent.

Source. OpenStax Statistics, §7.2, Example 7.2 — CC-BY

Example— 2· Election: probability of sample proportion

Problem. In an election, 42% of voters support candidate A ( $p = 0{,}42$ ). An exit poll interviews 1,000 voters at random. What is the probability that the sample proportion $\hat p$ is below 0.40?

Strategy. $\hat p$ is the mean of iid Bernoullis. By CLT, $\hat p \approx \mathcal{N}(p, p(1-p)/n)$ .

Resolution.

$\sigma_{\hat p} = \sqrt{0{,}42 \times 0{,}58 / 1000} = \sqrt{0{,}000244} \approx 0{,}01562$ .

$z = \frac{0{,}40 - 0{,}42}{0{,}01562} \approx -1{,}28.$

$P(\hat p < 0{,}40) = \Phi(-1{,}28) \approx 0{,}1003 \approx 10\%$ .

Verification. Difference of 2 percentage points with $\sigma \approx 1{,}6\%$ gives $z \approx -1{,}25$ ; values are consistent.

Source. OpenIntro Statistics, §5.4, Exercise 5.39, p. 224 — CC-BY-SA

Example— 3· Sample size for margin of error

Problem. An institute wants to estimate the average salary of engineers in SP with a margin of error $\pm$ R$ 200 at 95% confidence. Pilot survey: $s = 1,800$ reais. What is the minimum $n$ ?

Strategy. By CLT, 95% CI has semi-width $1{,}96\,\sigma/\sqrt{n}$ . Isolate $n$ .

Resolution.

$1{,}96 \times \frac{1800}{\sqrt{n}} = 200 \implies \sqrt{n} = \frac{1{,}96 \times 1800}{200} = 17{,}64 \implies n = 17{,}64^2 \approx 311{,}2.$

Round up: $n = 312$ .

Verification. $1{,}96 \times 1800/\sqrt{312} \approx 199{,}6 < 200$ . Correct.

Source. OpenStax Statistics, §7.3, Sample Size Calculation — CC-BY

Example— 4· Sum of insurance losses (actuarial modeling)

Problem. An insurer has 500 car policies. Each claim has a mean cost of R$ 3,000 and standard deviation of R$ 1,200 (exponential distribution). What is the probability that the total claims exceed R$ 1,560,000?

Strategy. $S_{500} = \sum X_i \approx \mathcal{N}(500 \times 3000,\; 500 \times 1200^2)$ by CLT (sum version).

Resolution.

$E[S] = 500 \times 3000 = 1,500,000$ ; $\text{Var}(S) = 500 \times 1200^2 = 720,000,000$ .

$\sigma_S = \sqrt{720,000,000} \approx 26,833$ .

$z = \frac{1,560,000 - 1,500,000}{26,833} \approx \frac{60,000}{26,833} \approx 2{,}237.$

$P(S > 1,560,000) = P(Z > 2{,}237) \approx 1 - 0{,}9873 \approx 1{,}3\%$ .

Verification. $\pm 2\sigma_S$ corresponds to $\pm$ R$ 53,666; $60,000 > 2\sigma_S$ , so less than 2.5% — consistent with 1.3%.

Source. OpenIntro Statistics, §4.4, p. 199–202 — CC-BY-SA

Example— 5· Berry-Esseen: quantifying approximation error

Problem. $X_i \sim \text{Bernoulli}(0{,}1)$ ; $\mu = 0{,}1$ , $\sigma^2 = 0{,}09$ , $E[|X-\mu|^3] = 0{,}9^3 \times 0{,}1 + 0{,}1^3 \times 0{,}9 = 0{,}0738$ . For $n = 100$ , what is the Berry-Esseen bound on the error $\sup_z |F_{Z_{100}}(z) - \Phi(z)|$ ?

Strategy. Apply the inequality directly with $C = 0{,}5$ .

Resolution.

$\rho = E[|X - \mu|^3] = 0{,}0738$ ; $\sigma^3 = 0{,}3^3 = 0{,}027$ .

$\text{Bound} \leq \frac{0{,}5 \times 0{,}0738}{0{,}027 \times \sqrt{100}} = \frac{0{,}0369}{0{,}27} \approx 0{,}137.$

The maximum approximation error is less than 13.7 percentage points — conservative bound, real error is much smaller.

Verification. For Bernoulli(0.1) with $n = 100$ , binomial vs. normal table shows real error of 1–2% — Berry-Esseen is loose as expected.

Source. Grinstead & Snell, Introduction to Probability, §9.3, p. 346–349 — GNU FDL

Exercise list

37 exercises · 9 with worked solution (25%)

Application 20Understanding 4Modeling 9Challenge 2Proof 2

Ex. 77.1Application
$X$ exponential with $\mu = 1$ and $\sigma = 1$ . Write the approximate distribution of $\bar X_{100}$ and calculate $\sigma_{\bar X}$ .
Solve online
Ex. 77.2Application
$X$ uniform on $[0, 1]$ . Determine $\mu$ and $\sigma^2$ , and write the approximate distribution of $\bar X_{50}$ by the CLT.
Solve online
Ex. 77.3ApplicationAnswer key
Roll 100 fair dice. Determine the approximate distribution of the sum $S_{100}$ , stating $E[S]$ and $\text{Var}(S)$ .
Solve online
Ex. 77.4Application
$X \sim \text{Bernoulli}(0{,}3)$ . Write the approximate distribution of $\bar X_{200}$ by the CLT and calculate the standard deviation of the sample proportion.
Solve online
Ex. 77.5Application
A population has $\mu = 50$ and $\sigma = 10$ . For $n = 25$ , calculate the standard deviation of $\bar X$ (as an integer).
Solve online
Ex. 77.6ApplicationAnswer key
Using the data from 77.5 ( $\mu = 50$ , $\sigma = 10$ , $n = 25$ ), calculate $P(\bar X > 53)$ .
Solve online
Ex. 77.7Application
With the same parameters ( $\mu = 50$ , $\sigma = 10$ , $n = 25$ ), calculate $P(\bar X < 47)$ .
Solve online
Ex. 77.8Application
$X$ with $\mu = 100$ , $\sigma = 20$ , $n = 100$ . Calculate $P(98 < \bar X < 102)$ .
Solve online
Ex. 77.9Application
Sum of 50 iid r.v. with $\mu = 5$ , $\sigma = 2$ . Calculate $P(S_{50} > 270)$ .
Solve online
Ex. 77.10Application
$X$ with $\mu = 10$ , $\sigma = 3$ . How many observations $n$ for a 95% CI with margin of error $\pm 0{,}5$ ?
Solve online
Ex. 77.11Understanding
When sample size $n$ is multiplied by 4, the standard deviation of $\bar X$ ( $= \sigma/\sqrt{n}$ ):
Solve online
Ex. 77.12Understanding
$X$ has a very skewed distribution (skewness = 3). For what size of $n$ is the CLT reasonable?
Solve online
Ex. 77.13Application
Grades with $\mu = 70$ , $\sigma = 15$ . Sample $n = 36$ . Calculate $P(\bar X > 75)$ .
Solve online
Ex. 77.14Application
With the same parameters as 77.13 ( $\mu = 70$ , $\sigma = 15$ , $n = 36$ ), calculate $P(\bar X < 65)$ .
Solve online
Ex. 77.15Application
With $\mu = 70$ , $\sigma = 15$ , $n = 36$ and $\bar X = 72$ , construct a 95% CI for $\mu$ .
Solve online
Ex. 77.16ApplicationAnswer key
Package weight: $\mu = 500$ g, $\sigma = 50$ g. Sample $n = 25$ . Calculate $P(\bar X > 520)$ .
Solve online
Ex. 77.17ApplicationAnswer key
With the parameters from 77.16 ( $\mu = 500$ g, $\sigma = 50$ g, $n = 25$ ), calculate $P(485 < \bar X < 515)$ .
Solve online
Ex. 77.18Application
Response time: $\mu = 50$ ms, $\sigma = 10$ ms. Mean of 100 measurements. What is the 95% SLA limit?
Solve online
Ex. 77.19Application
Roll a die 1,000 times. Calculate $P(\bar X > 3{,}6)$ .
Solve online
Ex. 77.20Application
Using the distribution of the sum $S_{1000}$ of 1,000 die rolls, calculate $P(S_{1000} > 3600)$ .
Solve online
Ex. 77.21Application
$X \sim \text{Exp}(1)$ ( $\mu = 1$ , $\sigma = 1$ ). Calculate $P(\bar X_{100} > 1{,}1)$ .
Solve online
Ex. 77.22ApplicationAnswer key
Election poll: $p = 0{,}40$ , $n = 1000$ . Calculate $P(\hat p > 0{,}43)$ .
Solve online
Ex. 77.23ModelingAnswer key
You hold 50 independent stocks; daily return of each: $\mu = 0{,}1\%$ , $\sigma = 2\%$ . What is the distribution of the average daily return of the portfolio?
Solve online
Ex. 77.24ModelingAnswer key
ML model: individual error $\sigma = 0{,}5$ . Calculate the standard deviation of the mean error over 1,000 predictions.
Solve online
Ex. 77.25Modeling
Determine the sample size to detect a 5% difference in proportions with $\alpha = 0{,}05$ and 80% power.
Solve online
Ex. 77.26Modeling
Monte Carlo estimate of $\pi$ : $n$ random points in the square $[0,1]^2$ , count those falling in the quarter-disk. What is the standard deviation of the estimate of $\pi$ as a function of $n$ ?
Solve online
Ex. 77.27Modeling
Batch of 500 parts: $\mu = 100$ g, $\sigma = 5$ g. Determine the distribution of the total mass $S_{500}$ .
Solve online
Ex. 77.28Modeling
Bus wait time: $\mathcal{U}[0, 30]$ min. Calculate $P(\bar T_{50} > 16)$ for the average wait of 50 passengers.
Solve online
Ex. 77.29Modeling
X-bar control chart with $n = 5$ . The control limits are $\bar X \pm 3\sigma_{\bar X}$ . Calculate the interval width in terms of process $\sigma$ .
Solve online
Ex. 77.30Modeling
Satisfaction survey: margin of error $\pm 3\%$ at 95% confidence, $p$ unknown. What is the minimum $n$ ?
Solve online
Ex. 77.31ModelingAnswer key
Call time: $\mu = 3$ min, $\sigma = 1{,}5$ min. 100 calls per hour. Determine the distribution of total time and calculate $P(\text{total} > 330\text{ min})$ .
Solve online
Ex. 77.32ChallengeAnswer key
A/B test: 10,000 visitors per variant; conversion rate A = 5%, B = 6%. Is the 1 percentage point lift statistically significant? Calculate the $z$ -value and $p$ -value.
Solve online
Ex. 77.33Understanding
Which of the following correctly describes the Central Limit Theorem?
Solve online
Ex. 77.34Understanding
Why does the classical Lindeberg-Lévy CLT not apply to the Cauchy distribution?
Solve online
Ex. 77.35Challenge
Simulate the CLT in Python for an exponential distribution with $\lambda = 1$ . Generate histograms of 10,000 sample means for $n \in \lbrace 1, 5, 30 \rbrace$ and compare visually with the theoretical normal curve.
Solve online
Ex. 77.36Proof
Sketch the proof of the CLT via characteristic function, indicating where each hypothesis (finite variance, iid) is used.
Solve online
Ex. 77.37Proof
Show that the CLT implies the Weak Law of Large Numbers: if $Z_n \xrightarrow{d} \mathcal{N}(0,1)$ , then $\bar X_n \xrightarrow{P} \mu$ .
Solve online

Sources

OpenIntro Statistics (4th ed) — Diez, Çetinkaya-Rundel, Barr · 2019 · CC-BY-SA. Primary source for exercises 77.2, 77.4, 77.8, 77.11, 77.14–17, 77.22–23, 77.25–26, 77.28, 77.30, 77.33–34.
OpenStax Statistics — Illowsky, Dean · 2022 · CC-BY. Source for exercises 77.1, 77.3, 77.5–7, 77.9–10, 77.12–13, 77.18–19, 77.21, 77.24, 77.27, 77.29, 77.31, 77.35 and examples 1–3.
Introduction to Probability (Grinstead-Snell) — Grinstead, Snell · Dartmouth · GNU FDL. Source for exercises 77.19–20, 77.26, 77.36–37 and example 5.