🥷 Hidden Markov model

=== a statistical ⏩ Markov model, in which the system being modeled is assumed to be a Markov process, with unobserved hidden states.==

Assumptions

• Markovianity → 💭 Markov Assumption (Language)
• Word independence
  • → the current state of the observed node O_t depends solely upon the current state of the unobserved variable
• The transition probabilities are independent of time

Formula (focus 🏷 POS-Tagging)

==$\arg {\max }_{t_{1,n}}{\prod }_{i=1}^n\left(P\left(w_i\mid t_i\right)P\left(t_i\mid t_{i-1}\right)\right)$==

• $w_i=$ current word
• $t_i=$ current tag
• $P\left(w_i\mid t_i\right)$ = frequency
  • = likelihood of word $w_i$ under condition tag $t_i$
• $P\left(t_i\mid t_{i-1}\right)$ = context
  • = likelihood of tag $t_i$ under condition previous tag $t_{i-1}$

How to estimate these probabilities?

→ use 🍔 Maximum likelihood estimation:

• $P\left(w_i\mid t_i\right)$ = $\frac{C\left(w_i,t_i\right)}{C\left(t_i\right)}$
• $P\left(t_i\mid t_{i-1}\right)$ = $\frac{C\left(t_i,t_{i-1}\right)}{C\left(t_{i-1}\right)}$

How to find the best “path”?

→ 🎻 Viterbi algorithm