📠 word2vec §

= a technique for learning word embeddings using shallow neural network

How? §

General approach:

1. Train on binary prediction task
   • Is w likely to show up near v?
2. Use learned classifier weights as word embeddings

Skip-Gram task §

• Approach: Predict if candidate word c is a neighbor
• Positive examples: target word t and context word c (+/- 2 word window)
• Negative examples: Random words from lexicon
• Train a 🏷 ML Classifiers to distinguish the two cases (→ 🚶 Logistic Regression)
  • Iteratively make the embeddings for a word:
    • more like the neighbors embeddings
    • less like the embeddings of other words
• Use the learned weights as the embeddings