GoConqr has a new look! Let us know what you think.

Sign up free

Flashcards

Exam Vocab

Alex F Natural Language Processing Univerität English 200 cards

Different types of text corpora

Cards in this deck (200)

  1. 1concordanceevery occurence of a word with context
  2. 2InterpreterProgram that will be running your python program
  3. 3Dispersion PlotDisplays location of word in the text, each instance represented as lines in the plot
  4. 4Token:Sequence of characters
  5. 5VocabularySorted list of all word types in a given text
  6. 6Lexical DensityNumber of distinct words divided by the total number of words
  7. 7Word TypeForm or spelling of the word independently of its specific occurence in a text. Unique item of vocabulary
  8. 8SlicingAccessing sublists/ manageable pieces of languages from large texts
  9. 9Frequency distributionFrequency of each vocabulary item in the text
  10. 10HapaxesWords that occur only once
  11. 11CollocationSequence of words that occur together unusually often
  12. 12Bigrams2-tuple of words
  13. 13Antecedentword, phrase or clause that is referred to by a pronoun or noun
  14. 14Word sense disambiguationwe want to work out which sense of a word was intended in a given context
  15. 157 challenges in automatic NLP UnderstandingWord sense disambiguation
    Anaphora and pronoun resolution
    Generating Language output
    Spoken Dialogue systems
    Semantic role labelling
    Textual entailment
    Text alignment
  16. 16Anaphora and pronoun resolutionfind the antecedent of a pronoun or nounphrase
  17. 17Generating Language Output Taskse.g. question answering and machine translation
  18. 18Spoken Dialogue Systemsprocess, understand and reply to a given user input
  19. 19Semantic role labellingidentify arguments of a predicate and their semantic role in a sentence
  20. 20Textual entailmentwhether a hypothesis is supported by some text
  21. 21Text alignmentgiven document in german and english, as well as a bilingual dictionnary, we can automatically pair up the sentences
  22. 22Turing testit is a measure of a machine's ability to demonstrate human-like
    intelligence. The test involves a human evaluator engaging in natural language
    conversations with both a human and a machine, without knowing which is which. If
    the evaluator is unable to reliably tell which is the machine and which is the human,
    the machine is said to have passed the Turing test.
  23. 23Gutenberg corpus25.000 free electronic books
  24. 24Web and Chat TextSmall collection of web text including different levels of jargon
  25. 25Brown Corpustext from 500 source categorized by genre, e.g. news, editorial etc
    tagged corpus
  26. 26Stylisticsstudying systematic differences between genres
    Brown corpus is useful for this
  27. 27Reuter Corpusnews docuemnts classified into 90 topics and grouped into training and test texts
  28. 28Inaugural address Corpuscollection of all US presidential addresses
  29. 29Annotated Text CorporaA text corpus containing linguistic annotations, representing POS tags, named entities, syntactic structures, semantic roles and so forth, e.g. brown corpus
  30. 30Corpora in other languages: udhrUniversal declaration of human rights in over 300 languages
  31. 31Different types of text corporaDifferent types of text corpora
  32. 32Conditional FrequencyCollection of frequency distributions for different conditions, e.g. category of the text. Visualized via tables, multi-line plots or cummulative word length distributions
  33. 33Generating random text with bigramsUsing a conditional frequency distribution to create a table of bigrams from a text, by treating each word as a condition, i.e. a freq dist over the words following a word. By appending the most likely word according to the distributions we can generate random text using this distribution.
  34. 34Lexical entryheadword (aka. Lemma) and additional information (such as POS and sense definition)
  35. 35Glossbrief definition or explanation of a word phrase, often provided in a different language to aid in understanding a text or speech
  36. 36Lemmabase or root form of a word, used to reduce the dimensionality of text data and imporve performance of NLP models by representing multiple inflected forms of a word with a single base form
  37. 37Homonymstwo distinct words having the same spelling
  38. 38stopwordshigh-frequency words that we sometimes want to filter out before further processing
  39. 39Pronouncing Dictionaryslightly richer kind of lexical ressource, table or spreadsheet containg a word with some additional properties per row
  40. 40Tabular lexiconSwadesh wordlists, lists of about 200 common words in several languages
  41. 41Shoebox and Toolbox Lexiconcollection of entries, consisting of one or more fields. Most fields are optional or repeatable, meaning this kind of lexicon can not be treated as table or spreadsheet
  42. 42WordNetsemantically oriented dictionary of English, similar to a traditional thesaurus but with a richer structure
  43. 43SynonymsTwo words are synonyms if they have the same meaning
  44. 44synsetsynonym set, collection of synonymous lemmas
  45. 45Unique beginners or root synsetsin WordNet these are words such as Entity, State, Event (very general concepts) without hypernyms
  46. 46Lexical relationsHypernyms, Meronyms, Holonyms and Hyponyms, as they establish relations between synsets
  47. 47Hyponymsa more specific concept, i.e. given the synset motorcar a (immediate) hyponym might be a motor
  48. 48Hypernymsmore abstract concept, e.g. given motorcar a hypernym may be vehicle
  49. 49Meronymsin wordnet: synset relation linking an item to the synset which it is part of, i.e. linking wheel to car
  50. 50Holonyms:in wordnet: synset relation linking an item and the things it contains, i.e. linking car to wheel
  51. 51Entailmentsrelationship between two statements or propositions in which one statment logically follows from or is implid by the other
  52. 52Antinomyrelationship between two opposites such as demand and supply
  53. 53Semantic similarityscore in the range of 0-1 based on the shortest path connecting the concepts in the hypernym hierarchy
  54. 54Tokenizationbreak up the string into words and punctuation
  55. 55Extracting text from binary formatsDownload the page
    strip HTML
    Trim useless content (e.g. introduction)
    Tokenize the text
    Normalize the words
    Build the vocab
  56. 56Text normalizationconverting text to lowercase, removing punctuation, stemming or lemmatization, removing stop words etc
  57. 57Sequencelists (mutable) and strings (immutable) are sequences in Python
  58. 58Code pointunique numerical identifier assigned to a character or symbol in a character encoding system. Unicode supports e06 code points
  59. 59Decodingtext in files will be in a particular encoding, decoding transforms that into Unicode
  60. 60Encodingtranslating from Unicode into a particular Encoding
  61. 61Glyphsfrom a Unicode perspective, characters are abstract entities that can be realized
  62. 62wildcard "."symbol matching any single character when looking for a specific format in the tokens
  63. 63Textonymstwo or more words entered with the same sequence of keystrokes (e.g. given T9 on a phone)
  64. 64Kleene closure"+" means one or more instances
    "*" zero or more instances
  65. 65Raw stringprefixing a string with r ignores all special characters inside
  66. 66Stemsbase form of a word after it has ben stripped of inflections (suffixes or prefixes)
  67. 67Popular StemmersPorter and Lancaster Stemmer
  68. 68LemmatizationWordNet Lemmatizer removes affixes only if the resulting word is in its dictionary. This additional checking process makes the lemmatizer slower than standard stemmers
  69. 69Segmentationdividing a corpus into different elements, such as sentences or tokens
  70. 70Word segmentationparticularly present in chinese. Processing of the spoken language, segmenting a continuous speech stream into individual words. In speech recognition the words or not known in advance!
  71. 71Objective functionscoring function based on the size of the lexicon and the amount of information needed to reconstruct the source text from the lexicon
  72. 72Simulated annealing in word segmentationoptimization method in used in word segmentation to find the optimal segmentation of a text by iteratively making small changes to the segmentation and accepting or rejecting them based on their likelihood
  73. 73String formatting expressionspermits to manipulate the format of a string, especially useful when the elements are printed in tables
  74. 74Conversion specifiers%s and %d symbols
  75. 75Format stringsstrings containing conversion specifiers
  76. 76Text WrappingOutput of our program is text-like, instead of tabular it will usually be necessary to wrap it so that it can be displayed conveniently
  77. 77Tupleimmutable sequences
  78. 78RecordCollection of different fields relating to some entity
  79. 79Generator expressiongenerates the required elements without preallocating a space in memory to the item
    e.g. finding item of max length in a text corpus (only store the longest word so far)
  80. 80Call-by-valuepython interprets function parameters as values
  81. 81LGB rulerule of function name resolution:
    Local Global Built-in
  82. 82Defensive programmingadding assertions to a program that helps you find logical errors
  83. 83Doctest blockIllustrates the use of the function and the expected output
  84. 84Lambda Expressiondefine functions as arguments to other functions
  85. 85Stack tracepinpoints the location of program execution at the time of the rror
  86. 86Debuggertool to monitor the execution of your program, specify line number where execution will stop and step through sections of code and inspect the value of variables
  87. 87regression testingmaintain a suite of test cases to spot where a change to the code has an unintended side effect of breaking something that used to work
  88. 88Divide and ConquerAttack a problem of size n by dividing it into two problems of size n/2, solve these problems and combine their results into a solution to the original problem
  89. 89Recursiondefine a function f, simplifying the problem and calling itself to solve one (or an easier amount of) instance of the same problem and then recombine this into a solution for the original problem
  90. 90Letter triedata structure that can be used for indexing a lexicon, one letter at a time. If trie contained a letter trie, then tried ['c'] would be a smaller trie which held all words starting with c
  91. 91Dynamic programmingUsed for problems containing overlapping subproblems. Instead of computing solutions to these subproblems repeatedly we store them in a lookuptable. Bottom-up and top down approaches are possible
  92. 92Bottom-upSolve smaller subproblems on the way to the original larger problems
  93. 93Top-down or memoizationbreak down a problem into smaller subproblems, saving their solutions in a table and recursively solving them to find the solution to the original problem
  94. 94Latent semantic analysisMethod using Numpy's singular value decomposiiton to help identify implicit concepts and relationships in a document collection
  95. 95NetworkXutility package for visualization when working with a wordNet dictionary or other complex networks
    defining and manipulating graphs
  96. 96Space-time trade-offin maintaining a vocabulary list. If you need to process an input text to check that all words exist in an existing vocabulary, the vocabulary should be stored as a set, not a list
  97. 97POS taggingclassifying words into their parts of speech and labelling them accordingly
  98. 98word classes/lexical categoriesparts of speech
  99. 99Tagsetcollection of tags used for a particular tasks
  100. 100POS taggerprocesses a sequence of words and attaches a POS tag to each word
  101. 101Adjectivesdescribe nouns and can beused as modifiers or as predicates
  102. 102Adverbsmodify verbs to specify time, manner or place of an event described by the verb. Can also modify adjectives
  103. 103Dictionarydata structure: key -> value, used to map arbitrary types
  104. 104types of Automatic taggingDefault tagger
    regular expression tagger
    lookup tagger
  105. 105Default taggerassigns the same tag to each token
  106. 106regular expression taggerassigns tags to tokens based on matching patterns. For instance words ending in "ed" might be the past participle of a verb and words ending in " 's " are posessive nouns
  107. 107Lookup Taggercompute the tag for the M most frequent words in a text. Knowing those you have already tagged a majority of the text (for M big enough)
  108. 108Backoffif the look-up table tagger does not find a query, it will use a default tagger
  109. 109Gold Standard datasetEvaluate the performances of a tagger one uses a corpus that was manually annotated and accepted as a standard against which the guesses of an automatic system are assessed. The tagger is regarded as being correct if the tag it guesses is the same as that gold standard tag
  110. 110Unigram Taggingfor each token, assign a tag that is most likely for that particular token. The training process involves inspecting the tag of each word and sotring the most likely tag for any word in a dictionary that is stored inside the tagger
  111. 111N-gram Tagginggeneralization of unigram tagger whose context is the current word together with the pos tags of the n-1 preceeding tokens. It takes the context into account to choose the most likely tag
  112. 112Precision/recall tradeoffMetric/Concept in machine learning and information retrieval that refers to the proportion of precision and recall
  113. 113PrecisionTrue positives / All positive tags
    how many items that we identified as relevant were actually relevant
  114. 114RecallTrue positives / Actual positives
    how many of the relevant items were identified as such
  115. 115Combining taggersCombine different taggers to improve results
  116. 116Out-of-vocabularylimit the vocab of a tagger to the most frequent n words, replacing everything else with "UNK"
  117. 117Confusion matrixEasy visualization of tagger performance that shows precision and recall in matrix form
    each cell [i,j] indicates how often label j was predicted when the actual label was i
  118. 118Brill Tagging (Transformation-based tagging)Deal with sparsity that arises when using n-gram tagger. Guess the tag of each word, go back and then fix mistakes. Does not count observations but compiles a list of tranformational correction rules
  119. 119How to determine category of a wordMorphological clues: suffixes
    Syntactic Clues: when do words occur
    Semantic clues: context of occurence
  120. 120Open Class Wordse.g. Nouns, can acquire new elements
  121. 121Closed Class Wordse.g. prepositions, determiners, modals, personal pronouns, can not acquire new elements
  122. 122Morphosyntactictaggers can learn information about the kind of morphological markings that words receive by their syntactic role
  123. 123Classification taskchoosing the correct class label for given input
  124. 124Supervised classifierclassifier trained on labeled data
  125. 125Feature extractorfunction that builds a dictionary containing relevant information about a given name
  126. 126Feature setmaps feature names to their values. Feature names are case sensitive are case sensitive strings providing short description of the feature, while the values can be anything, e.g. binary, integers
  127. 127OverfittingDescribes a model that performs very well on training data but does not generalize well. Common reasons are a too complex model on simple data, i.e. more features than samples (tends to rely on ideosyncrasies), or excessive training
  128. 128Error AnalysisSelect a development set containing training and dev-test data from the corpus. By analyzing performance on these sets we can perform error analysis
  129. 129Document Classificationwe can build classifiers hat will automatically tag new documents with appropriate category labels
  130. 130POS tagging Classifierclassifier that tags pos of a word using subword features and a classified corpus
  131. 131Exploiting Contextfeature detector examines context in which a word appears to determine which pos tag to assign
  132. 132Joint classifier modelschoose appropriate labelling for a collection of related inputs
  133. 133Sequence classifier modelsCan be used to jointly choose pos tags for all the words in a given sentence
  134. 134Hidden Markov ModelsMethod of sequence classification. Assigns scores to all possible sequences of pos tags and chooses the sequence whose overall score is the highest. They also look at both the inputs and the history of predicted tags. They generate a probability distribution over the tags.
  135. 135Maximum entropy marko models and linear-chain conditional random field modelsHidden Markov Models require that the feature extractor only looks at the most recent n tags. Given this restriction it is possible to utilize a dynamic programming approach to efficiently find the most likely tag sequence. For each consecutive word index i a score is computed for each possible current and previous tag.
  136. 136Sentence Segmentationcan be viewed as a classification task for punctuation: whenever we encounter a symbol that could end a sentence, such as a period or a questionmark, we have to decide whether it terminates the preceding sentence
  137. 137Identifying Dialogue Act TypesUtterances are a type of action performed by the speaker. These posts have all been labelled with one of 15 dialogue act types such as "Statement", "Emotion", "ynQuestion" and "Continuer". We can therefore use this data to build a classifier that can identify the dialogue act types for new instant messaging posts
  138. 138Recognizing Textual EntailmentThe task of determining whether a given piece of text T entails another text called the hypothesis, i.e. does our text imply this hypothesis
  139. 139AccuracyMeasures the percentage of inputs in the test set that the classifier labelled correctly
  140. 140True positivesrelevant items that were correctly identified as relevant
  141. 141True negativesirrelevant items that wer correctly identified as irrelevant
  142. 142false positivesirrelevant items that were incorrectly identified as relevant
  143. 143False negativesrelevant items that were incorrectly classified as irrelevant
  144. 144(k-fold) Cross Validationperform multiple evaluations on different test sets (folds), combine the scores from those evaluations to obtain an estimate for the generalization error
  145. 145Decision TreeSimple flowchart that selects labels for input values. It consists of decision nodes, which check feature values, assign labels by cascading down the tree and root nodes where the split starts
  146. 146Decision stumpdecision tree that consists of a single node that decides how to classify inputs based on a single feature
  147. 147Information gainmeasures how much more organized the input values become when we divide them up using a given feature
  148. 148Naive Bayes Classifiersbegins by calculating the prior probability of each label and then computes the likelihood score for each label, choosing the highest value
  149. 149Likelihood in Naive Bayes ClassifierP(label|features) = P(features,label)/P(features)
  150. 150Naive Bayes assumtion (indendence assumption)We assume that the Naive Bayes Classifier considers the features as totally independent from each other
  151. 151Smoothing techniquesExpected Likelihood Estimation adds 0.5 to each count(f,label)
    Heldout Estimation uses a heldout corpus to calculate the relationship between feature frequencies and the feature probabilities
  152. 152Binningconverting numeric features into binary (boolean) features
  153. 153Weights in the Naive Bayes Classifiernormal weights, initialized at some starting value and the final weights after training can be used as a measure of contribution for each feature
  154. 154Maximum Entropy ClassifierCalculates the likelihood of each label for a given input value. To do so it computes the joint-feature scores and multiplies the ones compatible with the input. It looks for the set of parameters that maximizes the total likelihood of the training corpus.
  155. 155Joint Featureeach combination of labels and features receives a parameter
  156. 156Maximum entropy principleAmong the distributions that are consistent with what we know, we should choose the distribution that maximizes entropy
  157. 157Generative ClassifierBuilds a model that predicts label and input occuring together.
    Joint probability of input and label
  158. 158Conditional Classifiersbuild models that predict conditional probability of a label given some input
    e.g. Maximum entropy classifier
  159. 159Modelling Linguistic Patternseither ML models or analytical models
  160. 160Structured DataRegular and predictable organization of entities and relationships
  161. 161Information ExtractionConvert the unstructured data of natural language sentences into structured data and then get meaning from the text
  162. 162Information Extraction - ProcessInformation Extraction - Process
  163. 163Named entity detectionIdentifying specific real world objects such as people, locations, organizations, products etc in text
  164. 164Relation DetectionSearch for specific patterns between pairs of entities that occur near one another in the text and use those patterns to build tuples recording relationships between the
  165. 165syntax errorsomething in the code is not conform with the syntax required by the language
  166. 166control structuree.g. if statement
  167. 167language understanding componentslanguage understanding components
  168. 168corporalarge bodies of linguistic data
  169. 169comparative wordlistExample for a tabular lexicon
  170. 170antonymyopposites
    supply and demand
    rush and linger
    horizontal and vertical
    staccato and legato
  171. 171multiword expressionsmall phrase whose meaning and other properties cannot be predicted from its words alone, e.g. part-of-speech
  172. 172mutable python typesList
    Dictionary
    Set
  173. 173Immutable python typesIntegers
    Strings
    Tuples
  174. 174Scope of a variableVariable defined inside the function body is only accessible inside that function
  175. 175consecutive classification aka greedy sequence classificationfind most likely class label for the first input and use that answer to find the best label for the next input and so on.
  176. 176F-Measurewhich combines the precision and recall to give a single score
    (2*Precsion*Recall)/(Precision+Recall)
  177. 177Entropyhigh if the sample has high intersample variance, i.e. there are a lot of different labels
    Defined as the sum of the probability of each libel times the log probability of that label
  178. 178PruningRemoving nodes from a decision tree that do not increase performance in order to decrease overfitting phenomena
  179. 179Zero countswhen a feature never occurs with a certain label a sample with said feature will never be attributed to that label
    Can be problematic if this phenomenom is specific to the training set and no general truth
  180. 180Double CountingAssumption of independence of feautres is not generally true and may lead to overestimations of contributions
    can be solved by applying weights to the features
  181. 181ChunkingBasic technique for entity recognition that segments and labels multitoken sequences into chunks
    Usually selects a subset of tokens
    Chunks do not overlap in the source text
  182. 182Noun Phrase chunkingsearch for chunks corresponding to noun phrases
    DT NN
    DT ADJ NN
    ...
  183. 183Chunk Grammarconsists of rules indicating how a sentence should be chunked
  184. 184Tag Patternsrules that make up a chunk grammar
  185. 185Chunking with regular expressionusing regexparser
    starting from a flat structure, applies the chunking rules in turn, updating the structure until all rules have been invoked
  186. 186chink"opposite of chunk"
    Instead of defining what goes into a chunk we define what to exclude from a chunk.
    E.g. excluding parts of a sentence that do not contribute to the meaning or are not part of a certain structure
  187. 187Tags vs TreesBoth ways to represent chunk structures.
    Tags give one full structure, while trees allow direct manipulation of each chunk as they are isolated constituents.
  188. 188IOB TagsInside Outside Beginning
    Used to represent chunk structures
  189. 189CoNLL-2000 Chunking corpusprovides text corpora and methods to work with chunked texts, represent them in tree diagrams, as well as training and evaluation models/chunkers.
  190. 190Evaluation and baselines of chunkersWe check IOB accuracy, Precision, Recall and F-Measure.
    A default chunker that creates no chunks, thus sets all samples to "O" is still correct for all tokens that would not be in any chunks (43.4% in the example)
  191. 191Tree traversalstandard to use a recursive function
  192. 192Named entity recognitionidentify all textual mentions of named entities
    - identifying boundaries of the named entiites
    - identifying its type
  193. 193Relation Extractionafter identifying named entities we want to extract the relations between them
  194. 194Linguistic Data and unlimited possibilitiesLanguage is ever changing and the amount of different sentence structures and therefore their meaning is unbounded
  195. 195Coordinate structuretwo phrases joined by a coordinating conjunction (and, but, or)
  196. 196Constituent Structurewords combine with other words to form units.
    Noticeable when trying to substitute words in a sentence
  197. 197ConstituentsComparable to the parent child relation in trees.
    Immediate constituents of Sentence S are verb phrase VP and noun phrase NP
  198. 198Context-Free GrammarDefine every constituent from the ground up. Sentences -> phrases -> word types -> specific words
  199. 199Structural Ambiguitymutliple structures can be created following the same grammar
    -> prepositional phrase attachement ambiguity
  200. 200Recursive grammarcategories appearing on the LHS of a production also appear on the RHS of a production
    i.e. definition calls itself

Information

Description
Univerität Natural Language Processing Flashcards on Exam Vocab , created by Alex F on 20/06/2023.
Created by
Alex F
Views
2
Rating

Feedback

What is it about?

What do you think of the new look?

How do you like it?