Statistical Regression

Statistical Regression

1. Involves the selection of a subset of predictors using a step-by-step method

2. Often used to:

   • Derive a smaller subset of 'good' predictors
   • Explore a new area of research
     • Gauging performance of the 'best' predictor(s)
     • Studying the nature of redundancies between predictor(s) and selecting the best subset

3. Forward Regression

   • Compared to standard regression, with less predictors you're more likely to get a more significant result with forward regression
   • Uses partial correlation (pr)
   • Might exclude some predictors that, in combination, contribute more
   • Might kick out the ones that give the most unique together

4. Partial Correlation

   • Is a measure of the unique contribution of X1 to the prediction of the unexplained part of Y
   • (pr)^2 of unselected predictors changes at each stage, depending on what is left unexplained
   • NOTE: Partial correlation can be high but it could be misleading if what is left unexplained is a small amount

5. Backward Regression

   • Uses semi-partial correlation (sr)
   • Occasionally, it might exclude the best, but redundant, predictor
   • Might kick out our best bi-variate predictor

6. Stepwise Regression

   • Recommended by most authors
   • It's a combination of forward and backward regression
     • 1. Start with empty equation
     • 2. Include best X if its contribution is p<.05
     • 3. Then exclude worse X if its contribution is p is not <.10
     • 4. Repeat steps 2 and 3
   • Often results produced by all three methods are the same

Images and files in the map