📘 Chapter 5 – Joint Probability Distributions

Plain English: It’s one rule (or table) that tells you the chance of X being some value and Y being some value at the same time.

Math: Discrete: p(x,y)=P(X=x, Y=y) with ∑∑ p(x,y)=1. Continuous: f(x,y) with ∬ f(x,y)\,dx\,dy=1.

Plain English: Add across to get totals for X, add down to get totals for Y.

Math: p_X(x)=∑_y p(x,y), p_Y(y)=∑_x p(x,y). (Continuous: integrate.)

Plain English: Add (or integrate) all the bits that sit in the rectangle/region A×B.

Math: Discrete: ∑_{x∈A}∑_{y∈B} p(x,y). Continuous: ∬_{A×B} f(x,y)\,dx\,dy.

Plain English: If every cell equals (row total × column total), they’re independent. If any cell fails, they’re not.

Math: Need p(x,y)=p_X(x)p_Y(y) (or f(x,y)=f_X(x)f_Y(y)) for all x,y.

Plain English: Average the X’s using the joint probabilities; same idea for Y.

Math: E[X]=∑_x∑_y x·p(x,y), E[Y]=∑_x∑_y y·p(x,y).

Plain English: It’s the average of the product X×Y. We use it to get covariance.

Math: E[XY]=∑_x∑_y xy·p(x,y); then Cov(X,Y)=E[XY]−E[X]E[Y].

Plain English: First get the average of X² (and Y²), then subtract (average of X)² (and Y)².

Math: E[X²]=∑∑ x²p(x,y), so Var(X)=E[X²]−(E[X])². Similar for Y.

Plain English: Look at the Y=y* column only; divide each cell by that column total.

Math: p(X=x|Y=y*)=p(x,y*)/p_Y(y*), where p_Y(y*)=∑_x p(x,y*).

Plain English: Look at the X=x* row only; divide each cell by that row total.

Math: p(Y=y|X=x*)=p(x*,y)/p_X(x*), where p_X(x*)=∑_y p(x*,y).

Plain English: After you restrict to the Y=y* column, average the X values using those conditional probabilities.

Math: E[X|Y=y*]=∑_x x·p(X=x|Y=y*).

Plain English: Average the conditional distributions using how often each Y happens.

Math: p_X(x)=∑_y p(X=x|Y=y)·p_Y(y). (Law of total probability.)

Plain English: Average the conditional averages to get the overall average.

Math: E[X]=E[E[X|Y]].

Plain English: Knowing Y tells you nothing about X; conditionals equal the marginals.

Math: p(X=x|Y=y)=p_X(x) for all x,y (and similarly for Y|X).

Plain English: Same idea: “slice” by Y=y and rescale that slice to sum to 1.

Math: f_{X|Y}(x|y)=f(x,y)/f_Y(y), where f_Y(y)=∫ f(x,y)dx.

Plain English: It tells you whether X and Y move together (+), opposite (−), or not in a clear linear way (~0).

Math: Cov(X,Y)=E[XY]−E[X]E[Y].

Plain English: It uses the units of X times the units of Y (e.g., dollars×hours), which makes it hard to compare across problems.

Math: Units = units(X)×units(Y).

Plain English: It’s the “scaled” version of covariance so it’s unit-free. It must be between −1 and +1.

Math: ρ=Corr(X,Y)=Cov(X,Y)/(σ_X σ_Y), with −1≤ρ≤1.

Plain English: Not usually. Zero correlation only guarantees independence for special families (like bivariate normal).

Math: In general ρ=0 ⇏ independence. For bivariate normal, ρ=0 ⇒ independence.

Plain English: Stretching X or Y (multiply by a or c) scales covariance; correlation stays the same except its sign flips if you multiply by a negative.

Math: If U=aX+b, V=cY+d: Cov(U,V)=ac·Cov(X,Y), Corr(U,V)=sign(ac)·Corr(X,Y).

Plain English: (1) Get the averages of X, Y, and XY. (2) Turn those into variances for X and Y. (3) Make covariance from E[XY], then divide by the product of SDs.

Math (discrete):

1. E[X]=∑∑ x p(x,y), E[Y]=∑∑ y p(x,y), E[XY]=∑∑ xy p(x,y).
2. E[X²]=∑∑ x² p(x,y), Var(X)=E[X²]−(E[X])² (same for Y).
3. Cov=E[XY]−E[X]E[Y], ρ=Cov/(√Var(X) √Var(Y)).

With raw data (not a table), Excel: =COVARIANCE.S(rangeX,rangeY), =CORREL(rangeX,rangeY).