NLE Keywords

NLE Keywords

1. Tokenisation

   • Convert string of chars into a sequence of tokens

2. Segmentation

   • Sequence of tokens into sequence of sentences

3. Stemming

   • Removing of affixes & suffixes from tokens

4. Lemmatisation

   • Morphological analysis - base form

5. POS Tagging

   • Label tokens with noun/verb/adjective/etc

6. Phrasal chunking

   • Identify subsequences of tokens that form phrasal units (sequence of words that have meaning) typically referring to entities or actions

7. Syntactic Analysis

   • Hierarchical analyse sentences into components i.e. subject/object/main verb/prepositional phrase
     • Phrase Trees/CFG/Dependency Parsing

8. Semantic Analysis

   • Express literal meaning of sentences using formal logical language
     • Lexemes, Lemmas & Word Senses
     • The meaning of individual words

9. Named entity recognition

   • Identify type of entity (sub task of information extraction that seeks to locate & classify elements in text into pre-defined categories such as the names of persons, organisations, locations, expressions of times. etc

10. Reference Resolution

    • link different references to the same entity

11. Word sense disambiguation

    • Using context to disambiguate use of word

12. Relation detection

    • Identify and classify relationships between entities

13. Event Detection

    • Identify, classify and temporarily order events

14. Topic identification

    • Identify words/phrases that convey a topic of document

15. Text Similarity

    • Measure relevance of a document to a query or of one document to another