Flashcards
Natural Language Engineering
Cards in this deck (17)
- 1Tokenisationconverting strings to tokens
- 2Segmentationsplitting into sentences
- 3Stemming
- 4Lemmatisation
- 5Part-of-speech tagginglabel tokens nouns/verbs/adjectives etc
- 6phrasal chunkingform phrasal units from tokens, typically refer to enitities and actions
- 7Syntactic analysishierarchical analyse sentences into components (subject/object/main verb)
- 8Semantic analysisexpress literal meaning
- 9Named entity recognistionidentify type of entity (person/institution/place/time)
- 10Reference resolutionlink difference references to the same entity
- 11Word sense disambiguationusing context to disambiguate use of a word
- 12Relation detectionidentify and classify relationships between entiities
- 13Event detectionidentify, classify and temporarily order events
- 14Topic identificationidentify words/phrases that relate to a topic
- 15Text similaritymeasure relevance of a document to a query, or of one document to another
- 16NLP pipeline
- 17Why is NLP hard?Many different ways of saying things
Hundreds of different dialects
Language is used in highly creative ways