Mean & Variance — Detailed Derivations (Chapter 3)

Uniform[a,b] — Integral Work

PDF: \(f(x)=\frac{1}{b-a},\; a\le x\le b\).

Mean

\(E[X]=\int_a^b x\frac{1}{b-a}dx=\frac{1}{b-a}\left[\frac{x^2}{2}\right]_a^b=\frac{a+b}{2}\)

Variance

\(E[X^2]=\frac{1}{b-a}\left[\frac{x^3}{3}\right]_a^b=\frac{b^2+ab+a^2}{3}\)

\(Var(X)=E[X^2]-(E[X])^2=\frac{(b-a)^2}{12}\)

Binomial(n,p) — Sum of Bernoullis

PMF: \(P(X=k)=\binom{n}{k}p^k(1-p)^{n-k}\). View \(X=\sum_{i=1}^n Y_i\), \(Y_i\sim \text{Bernoulli}(p)\), i.i.d.

\(E[X]=\sum E[Y_i]=np\), \quad \(Var(X)=\sum Var(Y_i)=np(1-p)\).

Geometric(p) — Detailed Derivations (failures before 1st success)

Definition used here: \(X\in\{0,1,2,\dots\}\) counts failures before the first success in i.i.d. Bernoulli(p) trials.
PMF: \(\displaystyle P(X=k)=(1-p)^k\,p,\; k=0,1,2,\dots\). Let \(q=1-p\).

Step 1 — Series Tools You Need

Step 2 — Mean \(E[X]\)

\(\displaystyle E[X]=\sum_{k=0}^{\infty} k\,P(X=k)=\sum_{k=0}^{\infty} k\,q^{k}p = p\sum_{k=0}^{\infty} k q^{k}.\)

Using \(\sum k r^{k}=\dfrac{r}{(1-r)^2}\) with \(r=q\):

\(\displaystyle E[X]= p\cdot \frac{q}{(1-q)^2}= p\cdot \frac{q}{p^2}=\frac{q}{p}=\frac{1-p}{p}.\)

\(\boxed{\;E[X]=\dfrac{1-p}{p}\;}\)

Step 3 — Second Moment \(E[X^2]\)

\(\displaystyle E[X^{2}] = \sum_{k=0}^{\infty} k^{2} P(X=k) = p\sum_{k=0}^{\infty} k^{2} q^{k}.\)

Using \(\sum k^{2} r^{k}=\dfrac{r(1+r)}{(1-r)^3}\) with \(r=q\):

\(\displaystyle E[X^{2}] = p\cdot \frac{q(1+q)}{(1-q)^{3}} = p\cdot \frac{q(1+q)}{p^{3}} = \frac{q(1+q)}{p^{2}}.\)

Step 4 — Variance and Standard Deviation

\(Var(X)=E[X^{2}]-(E[X])^{2} = \dfrac{q(1+q)}{p^{2}} - \left(\dfrac{q}{p}\right)^{2} = \dfrac{q}{p^{2}}.\)

\(\boxed{\;Var(X)=\dfrac{1-p}{p^{2}}\;}\)

Standard deviation: \(\displaystyle \sigma = \sqrt{\dfrac{1-p}{p^{2}}} = \dfrac{\sqrt{1-p}}{p}.\)

Step 5 — Alternative Parameterization (trials to first success)

Intuition: When \(p\) is small, the expected number of failures before a success grows like \(1/p\) and variability is large (order \(1/p^{2}\)).

Negative Binomial(r,p) — Detailed Derivations

PMF (failures before r-th success): \( \displaystyle P(X=k)=\binom{k+r-1}{k}(1-p)^k p^r,\; k=0,1,2,\dots\). Counts the total number of failures until the \(r\)-th success occurs in i.i.d. Bernoulli(p) trials.

Approach A — Sum of r Geometric(p)

Approach B — Probability Generating Function (PGF)

Approach C — Direct Series (factorial moments)

Summary: All three routes agree — \(E[X]=\dfrac{r(1-p)}{p}\), \(Var(X)=\dfrac{r(1-p)}{p^2}\).

Poisson(λ) — Detailed Derivations

PMF: \( \displaystyle P(X=k)=e^{-\lambda}\frac{\lambda^{k}}{k!},\; k=0,1,2,\dots \).

Approach A — Series with Index Shift

Approach B — PGF / MGF

Approach C — Direct Summation Check (optional)

Summary: All methods yield \(E[X]=\lambda\) and \(Var(X)=\lambda\).

Tip for students: After reading the derivations here, go back to your Excel N=10 simulations and check how your sample mean/variance compare to these results.