Counting words

Bag-of-words vectors and n-grams, TF-IDF weighting, and similarity, inverted indexes and ranking for search.

About 4 hours of study

Learning objectives

Lessons

  1. 1

    Bag of words and n-grams Intermediate

    Turn documents into vectors of word counts over a shared vocabulary, the bag-of-words model, and recover some word order with n-grams. See what a bag keeps, what it throws away, and why the vocabulary grows so fast.

  2. 2

    TF-IDF: weighting words Intermediate

    Weight each word by how often it appears in a document (term frequency) and how rare it is across the collection (inverse document frequency). Compute TF-IDF by hand and in Python, and use it to find a document's keywords.

  3. 3

    Text similarity and search Intermediate

    Measure how alike two texts are with Jaccard and cosine similarity, find candidate documents fast with an inverted index, and rank them against a query with TF-IDF vectors: the classic design of a search engine.

Module practice

Module quiz

1. How many trigrams does a sentence of 5 words have?
2. What is the vocabulary in a bag-of-words model?
3. A word appears in 2 of 4 documents. What is its idf with idf = ln(N / df)?
4. Two documents have count vectors [1, 0] and [3, 0]. What is their cosine similarity?
5. What does an AND query compute over an inverted index?
6. Why does BM25 limit the effect of repeating a word many times in one document?

‹ Text basicsClassic models ›