Flashcards
Natural Language Engineering Weeks 1-4
Cards in this deck (17)
- 1Describe the process of document feature extractionIt's the process where we identify the tokens and sentences that make up the document.
This identification can be achieved by sentence segmentation, tokenisation and canonicalisation (stemming or lemmatisation). - 2What is tokenisation?When we chunk text into a sequence of tokens.
- 3What are regular expressions?They are a convenient way of using single expressions to describe more than one possible string.
Example: colour?r - 4Regular Expressions Characters
- 5What is canonicalisation?Canonicalisation is the removal of an unwanted distinction.
- 6What is Morphology?The study of the way that words are built up out of smaller parts.
- 7What are morphemes?The smallest meaningful units of language. They are the building block of words.
- 8What are the three classes of morphemes?Roots(basic word form that is not inflected)
Stems (a word which may be inflected by further affixes)
Affixes (Prefixes, Postfixes) - 9What's morpheme good for?Spell-checking
Decoding word class (adjective, noun etc) and meaning
Canonicalizing text data (Stemming or lemmatisation) so we can reduce words to their simplest form - 10What are stemming and lemmatisation?
- 11What are examples of good morphological analysers?Porter Stemmer
Morphy Stemmer
Finite State Automata
Finite State Transducers - 12List 3 Document Classification ScenariosSentiment Analysis
Topic Relevance Detection
Document Filtering - 13What is sentiment analysis?Sentiment analysis is a process where it's determined whether a document expresses positive, negative or neutral sentiment about some entity.
- 14What is topic relevance testing?A process where it's determined whether a document concerns a topic that is of interest or not.
- 15Explain the Document filter classification scenarioFilter the document and determine where it conforms to some notion of "acceptability"
- 16How does classification based on word lists work?We can use hand-crafted vocabulary lists or we can derive word lists automatically?
- 17Why should you avoid classifying using hand-crafted lists?It's hard to build comprehensive lists.
It's hard to build balanced lists. People express themselves in different ways. Some specific words may be used in a different way and different occasions. The relevance of words are not always obvious.
Information
- Description
- Flashcards on Natural Language Engineering Weeks 1-4, created by Alexandros Parpas on 06/01/2015.
- Created by
- Alexandros Parpas
- Views
- 6
- Rating