GoConqr hat ein neues Design! Sag uns, wie es dir gefällt.

Kostenlos registrieren

Karteikarten

NLP

Carien Geerlings Computer Science Englisch 43 Karten

Karten in diesem Stapel (43)

  1. 1Content Word?Open Class / is meaningfull
  2. 2Function word?Closed Class / Explains structural/grammatical relationships
  3. 3Byte-Pair EncodingCombine characters iteratively and add to vocab to recognize new words
  4. 4Bender RuleName languages studied; Use multiple languages (not just English)
  5. 5Corporaa computer-readable collection of text or speech
  6. 6Token learnertakes a raw training corpus and induces a vocabulary, a set of tokens
  7. 7Token segmenterTakes a raw test sentence and segments it into the tokens in the vocabulary
  8. 8variable k in the BPE algorithmrepresents the number of merges that is performed / the number of new tokens in the vocabulary
  9. 9data statementspecifies properties of a dataset
  10. 10adjust weights to...?- size of error
    - contribution of error
    - learning rate
  11. 11Weighted averageaverage over classes, weighted by class size
  12. 12Macroaveragingcompute the performance for each class, and then average over classes
  13. 13Microaveragingcollect the decisions for all classes into a single confusion matrix, and then compute precision and recall from that table
  14. 14Bootstrappingrepeatedly drawing large numbers of smaller samples with replacement
  15. 15Representational harmscaused by a system that demeans a social group
  16. 16Model carddocuments a machine learning model with information
  17. 17Traditional tasks of language model- next word prediction
    - calculate probability of sequence
  18. 18ELMOBiDirectional LSTM
  19. 19Attention handles...long distance dependencies
  20. 20Self-attentionattends to token representation in the same sequence
  21. 21MLMMasked language modeling: predict masked word
  22. 22What is the other task that BERT is pre-trained on?Next Sentence Prediction
  23. 23How is input represented in BERT?Token embeddings + Sentence embeddings + positional embeddings
  24. 24Fine-tuning = ...Transfer Learning
  25. 25RNN problemerror propagation > vanishing gradients
  26. 26TreebanksSyntactically annotated corpora
  27. 27Benchmarkstandard point of reference for comparison
  28. 28Test for constituents- Substitution
    - Movement
  29. 29Connectednessstructure is complete
  30. 30AcyclicityStructure is hierarchical
  31. 31Single-headEvery word has at most 1 head
  32. 32Projective (tree)No crossing edges
  33. 33Hypothesized error causes- formalized and reproducable
    - applied to all instances
    - tested by counterfactual analysis
  34. 34PhonemesSmallest units; differentiate between words
  35. 35PhonesUnits of speech; Different articulation
  36. 36AllophonesArticulated differently; not differentiated between words
  37. 37Non-symbolic aspects of speech- Speaker dependent
    - Situation dependent
  38. 38Analog (wave) to discrete- Sampling
    - Quantization
  39. 39Spectral Representation- Windowing
    - Fourier Transform
  40. 40HMM models- Acoustic Model
    - Pronounciation lexicon
    - Language model
  41. 41Teacher ForcingCorrecting the output of the model that is used as input again, so that it does not go all the way off track
  42. 42Word Error RateInsertions + Deletions + Substitutions
    ------------------------------------------
    Length of ground truth
  43. 43—

Informationen

Beschreibung
Computer Science Karteikarten am NLP, erstellt von Carien Geerlings am 25/05/2021.
Erstellt von
Carien Geerlings
Aufrufe
1
Bewertung

Feedback

Worum geht es?

Wie gefällt dir das neue Design?

Dein Eindruck