📘 Section 12.2 – Hypothesis Tests in Multiple Linear Regression

🎯 What Are We Testing?

In multiple linear regression, hypothesis tests answer questions like:

Does the full set of predictors, taken together, explain any variation in the response?
Is a particular coefficient β_j equal to zero (or some other value)?
Does a group of predictors add useful information beyond the others?

We assume the usual regression model:

Y = β₀ + β₁x₁ + β₂x₂ + … + β_kx_k + ε, ε ~ N(0, σ²), independent

Here, k is the number of predictors, so there are p = k + 1 parameters (including the intercept), and n observations.

🧪 Global F-Test – “Is the Regression Useful?”

The test for significance of regression checks whether the model with predictors does better than a model with only an intercept.

Hypotheses

H₀: β₁ = β₂ = … = β_k = 0
H₁: β_j ≠ 0 for at least one j

We partition the total variability:

SST = SSR + SSE

SST: total sum of squares
SSR: regression (model) sum of squares
SSE: error (residual) sum of squares

F Test Statistic

F₀ = (SSR / k) ÷ (SSE / (n − p)) = MSR / MSE

Numerator df = k (one for each slope)
Denominator df = n − p = n − k − 1
Reject H₀ if F₀ > F_{α, k, n−p} or if the p-value is small.

ANOVA Table Structure

Source	SS	df	MS	F
Regression	SSR	k	MSR = SSR / k	F₀ = MSR / MSE
Error	SSE	n − p	MSE = SSE / (n − p)	–
Total	SST	n − 1	–	–

Interpretation: If H₀ is rejected, the model with predictors explains a significant portion of variability in Y (at least one slope is nonzero).

📊 R² and Adjusted R²

The global F-test is often paired with R² and adjusted R² to summarize overall fit.

R² = SSR / SST = 1 − SSE / SST

R² measures the proportion of total variability in Y explained by the regression. It always increases when you add more predictors.

To “penalize” unnecessary predictors, we use adjusted R²:

R²_adj = 1 − [SSE / (n − p)] ÷ [SST / (n − 1)]

R²_adj increases only if the new predictor reduces MSE.
Useful for comparing models with different numbers of predictors.

🧍 t-Tests for Individual Coefficients

Once the overall model is significant, we often ask whether individual predictors are helpful given that the others are already in the model.

General Hypotheses

H₀: β_j = β_j0
H₁: β_j ≠ β_j0

Test Statistic

t₀ = (β̂_j − β_j0) / se(β̂_j)

df = n − p
Reject H₀ if |t₀| > t_{α/2, n−p} (two-sided test) or based on the p-value.

Important Special Case

To test whether predictor x_j contributes to the model:

H₀: β_j = 0 vs. H₁: β_j ≠ 0

If we fail to reject H₀, x_j does not provide a significant partial contribution, given that the other predictors remain in the model.

🧩 Partial F-Tests for Groups of Predictors

Sometimes we want to test a subset of predictors at once (for example, a block of interaction or quadratic terms).

Idea: Compare Full vs. Reduced Models

Full model: includes all predictors under consideration.
Reduced model: sets the subset of coefficients to 0 (drops those predictors).

Extra sum of squares due to the subset:

SSR(extra) = SSR_full − SSR_reduced

Partial F Statistic

F₀ = [SSR(extra) / q] ÷ MSE_full

q = number of coefficients tested (size of the subset).
df_num = q, df_den = n − p for the full model.
Reject H₀ if F₀ > F_{α, q, n−p} or based on the p-value.

Special case: When q = 1 (testing a single coefficient), the partial F-test is exactly equivalent to the t-test:

F₀ = t₀²

🧠 Summary

Global F-test: checks whether the model with predictors is useful at all.
t-tests: check the partial effect of each individual predictor given the others.
Partial F-tests: check the joint effect of a group of predictors.
R² and adjusted R²: summarize how much variation in Y the model explains and help compare models.