Fichas
Regression - Chapter Five
Fichas del conjunto (16)
- 1Often there is more than one explanatory variable which we want to use to explain a response variable. We can extend the simple regression model to the multiple linear regression model denoted as ? Where i = ?, epsilon i ~ N(?,?), Cov(epsiloni,epsilonj) = ?Yi = B0 + B1xi1 + B2xi2 + ... + Bpxip + epsiloni
i = 1,2,...,n
epsiloni ~ N(0,sigma^2)
Cov(epsiloni,epsilonj) = 0 if i=/j - 2We know that our multiple linear regression line is linear as it is linear in its parameters B0, B1, B2,...,Bp but it does not have to be in the case of ... Give 2 examples of a multiple linear regression models Give an example of a multiple non-linear regression model... the explanatory variables.
1) Yi = B0 + B1x1i + B2x2i + epsiloni
2) Yi = B0 + B1x1i + B1x2i^2 + B3log(x3i) + epsiloni
Yi = B0 + e^(B1x1i + B2x2i) + epsiloni - 3Suppose that we have n observations then our multiple linear regression model becomes ...... Y1 = B0 + B1x11 + B2x12 + ... Bpx1p + epsilon1
Y2 = B0 + B1x21 + B2x22 + ... Bpx2p + epsilon2
....
Yn = B0 + B1xn1 + B2xn2 + ... + Bpxnp + epsilonn - 4This is usually written in matrix form which is denoted ... Which can then be neatly written as ... so its not in the big matrix form[Y1] = [1 x11 ... x1p]
[Y2] = [1 x21 ... x2p]
[... ] = [1 ... ... ...]
[Yn] = [1 xn1 ... xnp]
Y_ = XB_ + epsilon_ - 5What are the dimensions for the matrix? Where Y_ the ... vector's dimensions are ... Where X the ... matrix's dimensions are ... Where B_ the ... vector's dimensions are ... Where epsilon_ the ... vector's dimensions are ...[nx1] = [n x (p+1)][(p+1) x 1]+[n x 1]
... response ... n x 1
...design ... n x (p+1)
... parameter ... (p+1) x 1
error ... n x 1 - 6Why are the column of 1's necessary in the matrix? It is sometimes convenient to set xi_ as ... so that xi_ is the ...To give the constant term B0 for each observation
xi_ = 1
xi1
xi2, ...
xip
... vector of explanatory variables for the ith observation - 7Then we can express the design matrix as what?X = x1_^T
x2_^T
...
xn_^T - 8With multiple linear regression models how do we obtain the estimates for the Bs and sigma^2? The PDF for a univariate normal random variable is ?Exactly the same as the linear regression model, we form the likelihood and maximise.
f(y) = 1/sqrt(2*pi*sigma^2)exp[-(y-u)^2/2sigma^2] - 9If the observations are independent then the likelihood function is ?L(B_,sigma^2|y1,y2,...,yn) = [1/sqrt(2*pi*sigma^2)]^n*exp[-SIGMA(i=1,n)[yi-(B0+B1xip+...+Bpxip)]^2/2sigma^2] = [1/sqrt(2*pi*sigma^2)]*exp[-(SIGMA(i-1,n)[yi - xi_^TB_]^2/sigma^2]
- 10The log likelihood is then? SIGMA(i=1,n)[yi - xi_^TB_]^2 = ? Which we then let equal to what in order to simplify when differentiating?l(B_,sigma^2|y_) = -n/2log(2pi) - n/2log(sigma^2) - [SIGMA(i=1,n)[[yi-xi_^TB_]^2/2sigma^2]
(Y_ - XB_)^T(Y_ - XB_)
g(B_) - 11Now expanding g(B_) we get? Each term on the right hand side is what? So [Y_^T*X*B_] = ?g(B_) = (Y_ - XB_)^T(Y_ - X*B_)
= (Y_^T - B_^T*X^T)(Y_ - X*B_)
= [Y_^T*Y_] -[B_^T*X^T*Y_] -[Y_^T*X*B_] +[B_^T*X^T*X*B_]
A scalar
[Y_^T*X*B_] = (Y_^T*X*B_)^T = Y_*X^T*B_^T - 12Putting this into g(B_) we get? So partiald[g(B_)]/partiald[B_] = ? Setting this equal to 0 and putting a hat on B_ we get? So B_hat = ? Our estimate of B_hat will only exist if ?g(B_) = [Y_^T*Y_] - [2X^T*B_^T*Y_] + [X^T*B_^T*X*B_]
= -2X^T*Y_ + 2X^T*X*B_
0 = -2X^T*Y_ + 2X^T*X*B_hat
[X^T*X*B_hat] = [X^T*Y_] so
B_hat = [X^T*Y_]/[X^T*X] or [X^T*X]^(-1)*[X^T*Y_]
Provided that [X^T*X]^-1 exists - 13If we then let B_ equal B_hat and let yihat = xi_^T*B_hat in our log likelihood equation then the l(sigma^2|y_) = ? Replacing sigma^2 with ... and then differentiating we get ?l(sigma^2|y_) = -n/2log(2pi) - n/2log(sigma^2) - [SIGMA(i=1,n)[yi - yihat]^2/2sigma^2]
... τ ... partiald[τ|y_]/partiald[τ] = -n/2τ + SIGMA(i=1,n)[yi-yihat]^2/2τ^2 - 14On setting the equal to 0 and putting a hat on τ we get τhat = ? Similar to the linear regression model this is also a ... estimator so we end up with sigma^2hat = ?nτhat = SIGMA(i=1,n)[yi-yihat]^2 so
τhat = SIGMA(i=1,n)[yi=yihat]^2/n
... biased estimator ...
sigma^2hat = s^2 = SIGMA(i=1,n)[yi-yihat]^2/n-p-1 = RSS/n-p-1 - 15What is the vector of residuals e denoted as ?e = Y_ - Y_hat
= Y_ - X*B_hat
= Y_ - X[(X^T*X)^(-1)]*X^TY_
= (1-H)Y_
where H = X[(X^TX)^-1]X^T - 16H is known as the ... matrix because Y_hat = ? Therefore H converts Y_ to ... H^T = ?, i.e. it is ... HH = ? i.e. it is ... The diagonal values of H (hii) are called ? and are equivalent to ...... hat
... Y_hat = XB_hat = X[(X^TX)^-1]X^TY_ = HY_
... Y_hat
H^T = H ... symmetric
HH = H ... idompotent
Leverages ... the leverage calculated for a simple regression model
Información
- Descripción
- Fichas sobre Regression - Chapter Five, creado por Cara Wales el 25/05/2016.
- Creado por
- Cara Wales
- Visitas
- 0
- Valoración