Machine Learning for NLP

Machine Learning for NLP

1. MODELS

   • Generative model
     • Naive Bayes
     • P(⃗f , c) Joint distribution
   • Discriminative model
     • P ( c | ⃗f ) class given observation
     • discriminant functions—mapping from the observation to the class label without the probability.

2. Smoothing

   • additive smoothing/Laplace smoothing:add a small pseudocount α to each count

3. HMM

   • POS Tagging
     • Must include smoothing for unseen words

4. LSTMS

   • Unsupervised pre trained embeddigs

5. Named Entity Recognition

   • I/O or I/O/B Tagging for entities
   • MEM (Maximum Entropy Model)
     • Multinomial Logistic Regression
     • CRF Conditional Random Field is extension