🤦‍♂️ Naive Bayes §

= a simple (“naive”) 👮‍♂️ Supervised machine learning method for 🏷 Classifier based on bayes rule

Formal definition §

Assumptions §

• Bag of words assumption: Word position doesn’t matter
• Conditional independence: Feature probabilities are independent of the class

Formula §

$c_{NB}=\underset{c_j\in C}{\mathrm{argmax}}P\left(c_j\right){\prod }_{i\in positions}P(w_i\mid c_j)$

Calculating §

Using the formula above, $P(x_i\mid c_j)$ can be calculated:

1. $P(x_j\mid c_j)=\frac{P(x_j\&c_j)}{P(c_j)}$
2. 🍔 Maximum likelihood estimation: $\hat{P}\left(w_i\mid c_j\right)=\frac{\mathrm{count}\left(w_i,c_j\right)}{{\sum }_{w\in V}\mathrm{count}\left(w,c_j\right)}$
   • = fraction of times word $w_i$ appears among all words of topic $c_j$

Problems §

• Zero-probabilities: → smoothing (= add +1 to each word count)
• Unknown-words: → ignore them

👍 Pros §

• fast, low storage
• robust to irrelevant features
• good all-rounder (“Bayes has never failed me”)

📖 Example: §