Classic models

Naive Bayes text classification, lexicon and classifier approaches to sentiment with proper evaluation, and n-gram language models with smoothing and perplexity.

About 5 hours of study

Learning objectives

Lessons

  1. 1

    Naive Bayes text classification Intermediate

    Classify text with multinomial Naive Bayes: estimate class priors and word likelihoods from counts, smooth them with add-one, and add log probabilities to pick the most likely class. Build a spam filter from scratch.

  2. 2

    Sentiment analysis Intermediate

    Decide whether text is positive or negative: score words with a sentiment lexicon, handle negation and intensifiers, compare rules with trained classifiers, and measure the results with precision, recall and F1.

  3. 3

    Language models and smoothing Intermediate

    A language model gives every sentence a probability. Build bigram models from counts with sentence markers, see why unseen word pairs break maximum-likelihood estimates, fix them with add-one and add-k smoothing, measure models with perplexity and generate text.

Module practice

Module quiz

1. What does a Naive Bayes classifier multiply for each class?
2. What is recall for a class?
3. What does the end-of-sentence marker let a language model learn?
4. Precision is 0.5 and recall is 1.0. What is F1?
5. Why is add-one smoothing often too strong for a large vocabulary?
6. A lexicon scorer reads 'not bad at all' as negative. Which rule fixes it?

‹ Counting wordsStructure in text ›