multivariate regression

Page 1

single variate regression:

h-theta = theta-0 + theta-1 * x

multi-variate:

h-theta = theta-0 + theta-1 * x1 + theta-2 * x2 + theta-3*x3 +.....theta-n * xn

alternately:

h-theta = theta-0 * x-0 + theta-1 * x1 + theta-2 * x2 + theta-3*x3 +.....theta-n * xn

where x-0 = 1

in matrix form:

h= [theta]-Transpose * [x]

Cost function : multi-variate:

Page 2

objective of multi-variate linear regression would be :

choose theta-0, theta-1, theta -1 such that the cost function J(theta) will be a minimum;

Gradient descent algorithm:

Page 3

feature scaling:

to enable faster convergence: use feature scaling to bring the parameters into similar range:

-1

mean normalization:

(x1 - mean) /Range ; (Mean - X2)/range;

Page 4

Learning rate : alpha:

when using gradient descent : plot function J(theta) vs No of iterations

for sufficiently small alpha, this curve should be downward sloping;

smaller the alpha, slower the convergence;

if theta is large, then convergence may not occur and curve slopes upward

Page 5