Fichas
Regression - Chapter One
Fichas del conjunto (20)
- 1INTRODUCTION For the equation of the line y=B0+B1x a) What is B0? b) What is B1 When is the equation best used?a) B0 is the intercept
b) B1 is the slope
When a linear relationship is suspected - 2LEAST SQUARES REGRESSION The techniques "Least Squares" is used how? What does SSe stand for?To choose particular values of B0 and B1 that minimise SSe = SIGMA(i=1,n)[yi-B0-B1xi]^2
Sum of squares of the residuals - 3To find the values of B0 and B1 that minimise SSe what do you do? And what does this give you? To then find the minimum what then needs to be done to the derivatives?Differentiate SSe in respect to both B0 and B1
partialdSSe/partialdBo = -2SIGMA(i=1,n)[yi-B0-B1xi] and partialdSSe/partialdB1 = -2Sigma(i=1,n)[xi(yi-B0-B1xi)]
Set them equal to 0 and but B0 and B1 to B0hat and B1hat - 4To then find the minimum what then needs to be done to the derivatives? What does this give you?Set them equal to 0 and but B0 and B1 to B0hat and B1hat
0 = 2SIGMA(i=1,n)[yi-B0hat-B1hatxi] and
0 = 2SIGMA(=1,n)[xi(yi-B0hat-B1hatxi)] - 5SIGMA(i=1,n)[yi] = ? SIGMA(i=1,n)[B0hat] = ? SIGMA(i=1,n)[B1hatxi] = ?n*(ybar)
n*B0hat
n*B1hat*(xbar) - 60 = 2SIGMA(i=1,n)[yi-B0hat-B1hatxi] can be rewritten as ? 0 = 2SIGMA(i=1,n)[xi(yi-B0hat-B1hatxi] can be rewritten as ?0 = n*(ybar)-n*B0hat-n*B1hat*(xbar) i.e.
ybar - B0hat - B1hat*(xbar) = 0
SIGMA(i=1,n)[xi*yi]-(n*B0hat*xbar)-(B1hat*SIGMA(i=1,n)[xi^2]) - 7What do we therefore get for B0hat? What can we then do to get B1hat? What do we get for B1hat?B0hat = ybar - B1hat*xbar
Substitute what we have for B0hat into the other equation we have
B1hat = [SIGMA(i=1,n)[xi*yi]- n*xbar*ybar]/[SIGMA(i=1,n)[xi^2] - n*(xbar)^2] - 8What is Sxy? What is Sxx? What is Syy?Sxy = SIGMA(i=1,n)[(xi-xbar)(yi-ybar)] = SIGMA(i=1,n)[xi*yi]-n*xbar*ybar
Sxx = SIGMA(=1,n)[(xi-xbar)^2] = SIGMA(=1,n)[xi^2]-n*(xbar)^2 = SIMGA(=1,n)[xi^2]-[SIGMA(=1,n)[xi]]^2/n
Syy = SIGMA(=1,n)[(yi-ybar)^2] = SIGMA(=1,n)[yi^2]-n*(ybar)^2 = SIMGA(=1,n)[yi^2]-[SIGMA(=1,n)[yi]]^2/n - 9What must Sxx and Syy be? What it the equation for an estimated regression line?Positive
yhat = B0hat+B1hat*xbar - 10MAXIMUM LIKELIHOOD AND THE NORMAL MODEL What new equation do we use when we have to use maximum likelihood? E(epsiloni) = ? Var(epsiloni) = ? Cov(epsiloni,epsilonj) = ? Which parts of the equation are normally distributed?yi = B0+B1xi+epsiloni, where i=1,2,...,n
E(epsiloni) = 0
Var(epsiloni) = sigma squared
Cov(epsiloni,epsilonj) = 0 for all i not equal to j
epsiloni is normally distributed [epsiloni ~ N(0,sigma squared) also since B0, B1 and xi are constants, yi has normal dist... - 11E(yi|xi) = ? Var(yi|xi) = ? So yi ~ N(?,?)E(yi|xi) = E(B0+B1xi+epsiloni) = B0 + B1xi + E(epsiloni) = B0 + B1xi
Var(yi|xi) = Var(B0 + B1xi + epsiloni) = Var(epsiloni) = sigma squared
yi ~ N(B0+B1xi,sigma squared) - 12What is the pdf of yi? [f(yi|xi, B0, B1, sigma squared)f(yi|xi, B0, B1, sigma squared) = 1/[sqrt(2*pi*sigma squared)] exp[-[(yi-B0-B1xi)^2]/(2*sigma squared)]
- 13FORMING THE LIKELIHOOD What are our three unknown parameters in the model? As we assume independence in our observations, state the likelihood function to estimate the three unknown functions? Joint pdf is also the ...B0, B1 and sigma squared
L(B0, B1, sigma squared) = ∏(i=1,n)[f(yi|xi,B0,B1, sigma squared)
... product of the marginal pdfs - 14What is the log likelihood ? [l(B0, B1, sigma squared)]l(B0, B1, sigma squared) = SIGMA(i=1,n)log[L(B0, B1, sigma squared] = -n/2[log(simga squared)] -1/2sigma squared[SIGMA(i=1,n)[yi-B0-B1xi]^2] -n/2[log(2*pi)]
- 15MAXIMISING THE LIKELIHOOD From looking at our log likelihood how do we know what B0hat and B1hat already are and not sigma squared hat? What happens when we go to find the maximum likelihood estimator of sigma squared?B0 and B1 only depend on SIGMA(i=1,n)[yi-B0-B1xi]^2 which from this we already know that B0hat = ybar -B1hat*xbar and B1hat = Sxy/Sxx
B0 and B1 in the equation become B0hat and B1hat - 16How do we find the value of sigma squared that maximises the log likelihood? RSS = ?Differentiate the log likelihood with respect to sigma squared and then set it equal to 0 and out a hat on the estimate. First to make it simpler we substitute τ=sigma squared and denote RSS = SIGMA(i=1,n)[yi-B0hat-B1hatxi]^2
RSS = Residual Sum of Squared - 17What is our differential equation? What do we get when differentiating? What do we get for τhat? NEED TO REMEMBER!!! - Why is this estimator biased?partiald/partialdτ(-n/2[log(τ) -RSS/2τ -n/2[log(2*pi)])
-n/2*τhat + RSS/2*(τhat)^2 =0
τhat = RSS/n
E(τhat) = [(n-2)/n]*sigma squared - 18What is our final answer for sigma squaredhat?sigma squaredhat = 1/(n-2)*SIGMA(i=1,n)[yi-B0hat-B1hat*xi]^2
- 19Sometimes it is easier to leave it with RSS in the formula. Using B0hat = ybar -B1hat*xbar, what can RSS be rewritten as?RSS = SIGMA(i=1,n)[(yi-ybar)-B1hat(xi-xbar)]^2
Expanding then gives (yi-ybar)^2-2B1hat(xi-xbar)(yi-ybar)+[B1hat(xi-xbar)]^2
Which is equal to Syy - 2B1hat*Sxy -(B1hat)^2*Sxx
continued on next slide ... - 20—We know B1hat = Sxy/Sxx so
RSS = Syy -2(Sxy/Sxx)*Sxy - (Sxy/Sxx)^2*Sxx
RSS = Syy - 2(Sxy^2/Sxx)-(Sxy^2/Sxx)
RSS = Syy -(Sxy^2/Sxx)
Información
- Descripción
- Fichas sobre Regression - Chapter One, creado por Cara Wales el 13/05/2016.
- Creado por
- Cara Wales
- Visitas
- 1
- Valoración