Linear Regression

Linear Regression

1. Linear Regression Model

   • Coeffcients
   • Slope and intercept

2. OLS Estimator

   • Why we use it?
     • Estimate slope and intercept

3. Measures of Fit

   • R
     • Definition
     • Formula
     • Properties
       • Unitless
       • 0< R <1
   • SER
     • Definition
     • Formula

4. Least Square Assumption LSA

   • It's the key assumption to causal interpretation of beta1hat
     • 1. Conditional distribution of u given Xi has mean of zero
       • It this assumption holds then: Corr(u,Xi)=0
       • Controlled experiment vs. Observational Data
     • 2. Independently and Identically Distributed
       • Observations drawn by simple random sampling
     • 3. Large Outliers are Unlikely
       • X and Y have nonzero finite fourth moment
         • X and Y have finite kurtosis
         • Finite domain imply finite 4th moments

5. Sampling distribution of beta1hat

   • Beta1hat unbiased estimator of beta1
   • Variance of beta1hat
   • Consistency of beta1hat (LLN)
   • Under CLT beta1hat is approximated by a normal distribution

6. Regression with binary regressor (X=0,1)

   • OLS estimator is the difference of means analysis

7. Hypothesis Testing

   • Compute SE of beta1hat
   • Compute t-statistic
     • Reject Ho if:
       • p-value < significance level (0.05)
       • t-statistic (act) > critical value (1.96)

8. Confidence intervals (estimator +/- 1.96xSE(beta1hat)

9. Homoeskedasticity vs. Heteroskedasticity

   • New formula for SE
   • Meaning
   • Differences
     • Resulting t-statistic