VL11 NL1




How does ๐Ÿ“  word2vec work (general)?#card

General approach:

  1. Train on binary prediction task
    • Is w likely to show up near v?
  2. Use learned classifier weights as word embeddings

List the steps of the skip-gram task (word2vec)#card

  • Approach: Predict if candidate word c is a neighbor
  • Positive examples: target word t and context word c (+/- 2 word window)
  • Negative examples: Random words from lexicon
  • Train a ๐Ÿท ML Classifiers to distinguish the two cases (โ†’ ๐Ÿšถ Logistic Regression)
    • Iteratively make the embeddings for a word:
      • more like the neighbors embeddings
      • less like the embeddings of other words
  • Use the learned weights as the embeddings