Machine Learning

Annotations:

• P(C|x) = P(C) * p(x|C) / p(x)    

1. posterior p(C | x)

   Annotations:

   • = prior * likelihood / evidence
   1. prior P(C = i)

      Annotations:

      • probability of being class i 
   2. likelihood p(x|C)

      Annotations:

      • conditional probability, probability of event x given being class C 
   3. evidence p(x)

      Annotations:

      •  marginal probability probabilty of event x
2. Risk function
3. Reject class
4. Discriminant Functions g(x)
5. Associative Rule
   1. Support(X,Y) ≡ P(X,Y)
   2. Confidence(X → Y) ≡ P(Y|X)
   3. Lift(X → Y) = P(X,Y) / (P(X) * P(Y))

      Annotations:

      • = P(Y|X) / P(Y) kind of descripting relationship btw X and Y               

1. Complexity

   Annotations:

   • [Underfitting] Too low complexity, it may give high probability of false positive.
2. amount of data points
3. Generalisation error

   Annotations:

   • [Overfitting] Hypothesis class is too complicated and fit the training data points. However, the new data points may match the hypothesis and the classification error may raise.

1. g(x|θ)

   Annotations:

   •  where g(·) is the model, x is the input, and θ are the parameters 
2. Loss function, L(·)
3. E(θ|X) =
4. θ∗ =argminE(θ|X)