📘 Section 11.10 – Logistic Regression

🔍 1. Why Logistic Regression?

• Used when the dependent variable is binary (e.g., success/failure, yes/no).
• Linear regression assumes constant variance and normal errors—not true for binary Y.
• Logistic regression keeps predictions between 0 and 1 using a sigmoid (S-shaped) function.

🚫 2. Problems with Linear Regression for Binary Response

• Error terms are not normal—only two outcomes (0 or 1).
• Error variance is not constant: Var(Yi) = πi(1 − πi).
• Predicted values can fall outside the [0, 1] interval.

🔢 3. Logistic Function Derivation

Let π(x) = P(Y = 1 | x), the probability of success. The logistic function ensures predicted probabilities stay within [0, 1]:

π(x) = exp(β₀ + β₁x) / [1 + exp(β₀ + β₁x)]
     = 1 / [1 + exp(−(β₀ + β₁x))]
  

📉 4. Odds and Log-Odds

The odds of success are:

odds = π(x) / [1 − π(x)] = exp(β₀ + β₁x)

Taking the log gives the logit (log-odds):

log(odds) = log[π(x)/(1−π(x))] = β₀ + β₁x

📖 5. Interpretation of Coefficients

• β₁: change in log-odds for a one-unit increase in x.
• exp(β₁): odds ratio → how odds change when x increases by 1.
• If exp(β₁) = 0.84, then a 1-unit increase in x reduces odds by 16%.

📈 6. Graph: Logistic Curve (O-Ring Example)

Fitted model:
P(Y=1) = 1 / [1 + exp(−(10.875 − 0.17132 × Temp))]

📊 7. Estimation via Maximum Likelihood

• Parameters are estimated by maximizing the likelihood of observing the sample.
• The log-likelihood function is nonlinear and requires iterative methods.
• Common algorithms include Newton-Raphson and Fisher Scoring.
• Software (R, Stata, Python) reports coefficients, standard errors, and p-values.

✅ 8. Summary

• Use logistic regression when Y is binary (0 or 1).
• Predictions stay within [0, 1] due to the S-curve.
• Coefficients are interpreted via log-odds or odds ratios.
• Evaluate model fit using deviance, pseudo R², and visual plots.