week 7

Page 1

support vector machine:

variant of the logistic regression :

minimize the cost of the logistic regression with a minor variation

what is cost of logistic regression model?

A + Lambda*B

minimize it and identify the thetas;

variation:SVM

C*A + B

Page 2

support vector machine is also called large margin classifier;

mathss:

U'V :

projection on vector V on U * magnitude of U

Theta'*X = project of X on theta

Page 3

Kernels:

given x , identify new features based on the landmarks l1, l2 , l3

theta0 + theta1*f1 + theta2*f2 + theta3*f3

f1 = similiarity(x,l1);

f2 = similarity (x,l2)

Gaussian Kernels

predict 1 : when : theta0 + theta1*f1 + theta2*f2 + theta3*f3 > = 0

predict 0 : when : theta0 + theta1*f1 + theta2*f2 + theta3*f3 < 0

Page 4

given , compute features f E R^m+1

predict y = 1, theta' * f >= 0 ;

Page 5

SVM Parameters:

large C : small Lambda : Lower bias ; high variance

small C : large Lambda : Higher bias ; low variance

sigma^2 : large : Features vary more smoothly : high bias ; low variance

sigma^2 : small : features vary abruptly : low bias high variance;