Flashcards
NLU
Cards in this deck (14)
- 1Name 8 NLP statistical methodsHeuristic Approaches
Nearest neighbour
Decision Trees
Support vector machines
Naive Bayes
Maximum Entropy Models
PCFGs, HMMs
Conditional random fields - 2Name 4 probabilistic NLP methodsNaive Bayes
Maximum Entropy Models
PCFGs, HMMs,
Conditional random fields - 3Describe what statistical methods involveFunction(s) computed from data
counts, means, probabilities, threshold sets using development set - 4Describe what probabilistic methods involveDefining a probabilistic model describing the distribution of data D
-Observed variables e.g. word string
-Hidden variables e.g. tag sequence
-Model parameters θ e.g. P(x=cat|NN) - 5Generative model formula
- 6Discriminative model formula
- 7Difference between maximum likelihood estimation and maximum conditional likelihood estimation in terms of probability spaceMLE: makes (x,y) as likely as possible whereas MCLE makes (x,y) as likely as possible relative to other pairs with the same X value
- 8Describe the supervised paradigm applies in NLPTraining: Labelled training data, Statistics, Machine Learning System, Prediction procedure
Evaluation: New unlabelled data, statistics, prediction procedure (from training), labelled output - 9Describe the unsupervised paradigm in NLP systemsUnlabelled data, statistics, machine learning system, labelled output and prediction procedure
- 10Give 4 principles of human language processing (from linguistics, pyschology) that can inform engineering solutions1) Humans learn language from exposure to linguistic data
2) Human learning is mostly unsupervised
3) Human language processing is often heuristic and approximative
4) It operates with limited memory and input data - 113 examples of how human language processing can inform NLPhuman parsing is incremental (structures are built word by
word): constrains possible parsing algorithms;
human lexical processing is subject to priming (words with
similar meanings are processed faster): constrains possible
semantic representations;
human language learning uses limited input (about 350M
words during lifetime): constraints training set size. - 12Describe the HMM approach to POS taggingT! = argmax P(T|W)
= argmax P(W|T)P(T)
= argmax P(cur_word|cur_tag)P(cur_tag|prev_tag)
Optimises likelihood P(W|T), estimate posterior by combining the likelihood and the prior p(T) - 13Describe the Maximum Entropy Markov Model approach to POS taggingT! = argmax P(T|W)
= argmax product of P(cur_tag | cur_word, prev_word) - 14Give one advantage of the Maximum Entropy Markov Model (discriminative) approach to POS tagging compared to HMM (generative) approach to POS taggingMEMM can condition on any useful feature of the input observation. This is not possible in the HMM because the HMM is likelihood based and hence it would need to compute the likelihood feature of the observation
Information
- Description
- Flashcards on NLU, created by wrennie on 21/04/2014.
- Created by
- wrennie
- Views
- 17
- Rating