Machine Learning

Machine Learning

1. Prediction

   • Most common ML application
   • Types
     • Regression
       • Output y belongs to R, set of real numbers 
     • Classification
       • Output y belongs to a set of specific, possible outcomes e.g. {yes, no}, {0,1,2,....9}
   • Learning task
     • Given value of an input x, make a good prediction of output y, denoted by y hat. x: scalar or vector x = (x1, x2, .... xp) 'p' features y: scalar or vector
     • Supervised learning
       • Given a training se of N data points, learn a prediction function f:x->y such that given a new x, f can accurately predict the corresponding y.
       • Linear model
         • Error function
         • Hyper-parameters
           • Lambda - regularization coeff
             • Model selection
               • For different values of hyper-param (HP) - train the model - compute the perf in valid set
               • Pick val of HP that has best valid perf
               • Compute test perf for model with chosen value of HP
           • M - deg of polynomial
         • Overfitting
           • Sol 3: Model selection for based on M
           • Sol 2: Regularization
           • Sol 1: Add more data points
           • Checking for it: Use separate test set
         • Classification
           • M1: Linear model
             • Closed form solution
           • M2: k-Nearest Neighbour (k-NN)
             • Average of classification values of k closest neighbours
             • k - #nearest neighbours
     • Unsupervised learning

Images and files in the map