Fichas
NLP
Fichas del conjunto (43)
- 1Content Word?Open Class / is meaningfull
- 2Function word?Closed Class / Explains structural/grammatical relationships
- 3Byte-Pair EncodingCombine characters iteratively and add to vocab to recognize new words
- 4Bender RuleName languages studied; Use multiple languages (not just English)
- 5Corporaa computer-readable collection of text or speech
- 6Token learnertakes a raw training corpus and induces a vocabulary, a set of tokens
- 7Token segmenterTakes a raw test sentence and segments it into the tokens in the vocabulary
- 8variable k in the BPE algorithmrepresents the number of merges that is performed / the number of new tokens in the vocabulary
- 9data statementspecifies properties of a dataset
- 10adjust weights to...?- size of error
- contribution of error
- learning rate - 11Weighted averageaverage over classes, weighted by class size
- 12Macroaveragingcompute the performance for each class, and then average over classes
- 13Microaveragingcollect the decisions for all classes into a single confusion matrix, and then compute precision and recall from that table
- 14Bootstrappingrepeatedly drawing large numbers of smaller samples with replacement
- 15Representational harmscaused by a system that demeans a social group
- 16Model carddocuments a machine learning model with information
- 17Traditional tasks of language model- next word prediction
- calculate probability of sequence - 18ELMOBiDirectional LSTM
- 19Attention handles...long distance dependencies
- 20Self-attentionattends to token representation in the same sequence
- 21MLMMasked language modeling: predict masked word
- 22What is the other task that BERT is pre-trained on?Next Sentence Prediction
- 23How is input represented in BERT?Token embeddings + Sentence embeddings + positional embeddings
- 24Fine-tuning = ...Transfer Learning
- 25RNN problemerror propagation > vanishing gradients
- 26TreebanksSyntactically annotated corpora
- 27Benchmarkstandard point of reference for comparison
- 28Test for constituents- Substitution
- Movement - 29Connectednessstructure is complete
- 30AcyclicityStructure is hierarchical
- 31Single-headEvery word has at most 1 head
- 32Projective (tree)No crossing edges
- 33Hypothesized error causes- formalized and reproducable
- applied to all instances
- tested by counterfactual analysis - 34PhonemesSmallest units; differentiate between words
- 35PhonesUnits of speech; Different articulation
- 36AllophonesArticulated differently; not differentiated between words
- 37Non-symbolic aspects of speech- Speaker dependent
- Situation dependent - 38Analog (wave) to discrete- Sampling
- Quantization - 39Spectral Representation- Windowing
- Fourier Transform - 40HMM models- Acoustic Model
- Pronounciation lexicon
- Language model - 41Teacher ForcingCorrecting the output of the model that is used as input again, so that it does not go all the way off track
- 42Word Error RateInsertions + Deletions + Substitutions
------------------------------------------
Length of ground truth - 43—
Información
- Descripción
- Computer Science Fichas sobre NLP, creado por Carien Geerlings el 25/05/2021.
- Creado por
- Carien Geerlings
- Visitas
- 1
- Valoración