Machine Learning

Machine Learning

1. Supervised Learning

   • Vapnik-Chervonenkis (VC) Dimension
   • Classification
     • Bayesian Decision Theory
       • P(C|x) = P(C) * p(x|C) / p(x)    
       • posterior p(C | x)
         • = prior * likelihood / evidence
         • prior P(C = i)
           • probability of being class i 
         • likelihood p(x|C)
           • conditional probability, probability of event x given being class C 
         • evidence p(x)
           •  marginal probability probabilty of event x
       • Risk function
       • Reject class
       • Discriminant Functions g(x)
       • Associative Rule
         • Support(X,Y) ≡ P(X,Y)
         • Confidence(X → Y) ≡ P(Y|X)
         • Lift(X → Y) = P(X,Y) / (P(X) * P(Y))
           • = P(Y|X) / P(Y) kind of descripting relationship btw X and Y               
   • Regression
   • Modeling
     • Triple tradeoff
       • Complexity
         • [Underfitting] Too low complexity, it may give high probability of false positive.
       • amount of data points
       • Generalisation error
         • [Overfitting] Hypothesis class is too complicated and fit the training data points. However, the new data points may match the hypothesis and the classification error may raise.
     • Math notation
       • g(x|θ)
         •  where g(·) is the model, x is the input, and θ are the parameters 
       • Loss function, L(·)
       • E(θ|X) =
       • θ∗ =argminE(θ|X)