Counting words
Bag-of-words vectors and n-grams, TF-IDF weighting, and similarity, inverted indexes and ranking for search.
About 4 hours of study
Learning objectives
- Build count vectors and n-grams
- Weight words with TF-IDF
- Index, match and rank documents for a query
Lessons
- 1
Bag of words and n-grams Intermediate
Turn documents into vectors of word counts over a shared vocabulary, the bag-of-words model, and recover some word order with n-grams. See what a bag keeps, what it throws away, and why the vocabulary grows so fast.
- 2
TF-IDF: weighting words Intermediate
Weight each word by how often it appears in a document (term frequency) and how rare it is across the collection (inverse document frequency). Compute TF-IDF by hand and in Python, and use it to find a document's keywords.
- 3
Text similarity and search Intermediate
Measure how alike two texts are with Jaccard and cosine similarity, find candidate documents fast with an inverted index, and rank them against a query with TF-IDF vectors: the classic design of a search engine.
Module practice
- NLP: List the n-gramsEasy
- NLP: Bag-of-words count vectorsMedium
- NLP: The most common n-gramsHard
- NLP: Document frequencyEasy
- NLP: TF-IDF of one documentMedium
- NLP: Keywords by TF-IDFHard
- NLP: Jaccard similarity of two sentencesEasy
- NLP: An inverted index with AND queriesMedium
- NLP: Rank documents by TF-IDF cosineHard