📘 Section 12.6 – Aspects of Multiple Regression Modeling

📊 Example: Student GPA Model

We are modeling a student's GPA based on:

• x₁: Average study hours per week
• x₂: Hours of sleep per night
• x₃: Number of courses taken
• x₄: Part-time job hours per week

Suppose the estimated regression model is:

🔎 12.6.1 Multicollinearity

• Occurs when predictor variables are highly correlated
• Can inflate standard errors of coefficients
• Check Variance Inflation Factor (VIF): VIF > 10 is problematic

🎯 12.6.2 Use of Dummy Variables

Convert categorical variables (e.g., major = {STEM, non-STEM}) into 0/1 variables.

Example: STEM = 1 if STEM major, 0 otherwise.

📀 12.6.3 Model Selection – Mallow’s Cp

• Mallows' Cp is used to assess subset models
• Choose the model where Cp ≈ p (number of parameters)

⚙️ 12.6.4 Adjusted R²

Adjusted R² accounts for the number of predictors:

Use this to avoid overfitting – it can decrease if you add irrelevant predictors.

💡 Summary

• Check for multicollinearity using VIF
• Use dummy variables for categorical predictors
• Apply Mallows’ Cp for model selection
• Evaluate models using Adjusted R²
• Monitor influential observations with Cook’s Distance

📊 GPA Student Example Continued

Suppose we want to model a student's GPA (y) based on:

• x₁ = Hours studied per week
• x₂ = Number of courses
• x₃ = Hours of sleep
• x₄ = Dummy variable for part-time job

Model:

• Run full model → check VIF
• Use Adjusted R², C_p to choose reduced model
• Examine residual plots
• Use Cook’s Distance to identify influential students

📝 Final Checklist – How to Approach Multiple Regression Carefully

1. Fit Full Model using OLS: Y = Xβ + ε
2. Assess Goodness of Fit: Look at R² and Adjusted R²
3. Check Residuals: Look for nonlinearity or outliers
4. Check Multicollinearity: Compute VIF for each predictor
5. Model Simplification: Use Mallows’ Cp and Adjusted R²
6. Detect Influential Observations: Use Cook’s Distance (D > 1 is a warning)
7. Use Dummy Variables: For categorical features
8. Report Final Model: Include coefficients, R²_adj, and diagnostics
9. Validate: Use test data or cross-validation if possible
10. Interpret Carefully: Emphasize practical significance, not just p-values