🥷 Hidden Markov model §

= a statistical ⏩ Markov model, in which the system being modeled is assumed to be a Markov process, with unobserved hidden states.

Assumptions §

• Markovianity → 💭 Markov Assumption (Language)
• Word independence
  • → the current state of the observed node O_t depends solely upon the current state of the unobserved variable
• The transition probabilities are independent of time

Formula (focus 🏷 POS-Tagging) §

$\arg {\max }_{t_{1,n}}{\prod }_{i=1}^n\left(P\left(w_i\mid t_i\right)P\left(t_i\mid t_{i-1}\right)\right)$

• $w_i=$ current word
• $t_i=$ current tag
• $P\left(w_i\mid t_i\right)$ = frequency
  • = likelihood of word $w_i$ under condition tag $t_i$
• $P\left(t_i\mid t_{i-1}\right)$ = context
  • = likelihood of tag $t_i$ under condition previous tag $t_{i-1}$

How to estimate these probabilities? §

→ use 🍔 Maximum likelihood estimation:

• $P\left(w_i\mid t_i\right)$ = $\frac{C\left(w_i,t_i\right)}{C\left(t_i\right)}$
• $P\left(t_i\mid t_{i-1}\right)$ = $\frac{C\left(t_i,t_{i-1}\right)}{C\left(t_{i-1}\right)}$

How to find the best “path”? §

→ 🎻 Viterbi algorithm