Natural Language Processing
Tokens, regular expressions, TF-IDF, search, Naive Bayes, sentiment, language models, HMM tagging, entities, spelling, embeddings, BPE and attention, all in plain Python.
- Level
- Intermediate
- Time
- About 22 hours
- Modules
- 5
- Lessons
- 15
- Exercises
- 45
Natural language processing (NLP) lets computers read, sort, search and write human language: spam filters, search engines, translation, voice assistants and chatbots all depend on it. This course builds its core ideas step by step: cleaning and tokenizing text, regular expressions, bag-of-words and TF-IDF, similarity and search, Naive Bayes and sentiment analysis, n-gram language models, part-of-speech tagging with hidden Markov models, named entities, edit distance and spelling correction, word embeddings, byte-pair encoding and attention.
Everything is plain Python that runs in your browser, with no libraries to install, on texts small enough to check by hand. Each lesson also shows how professionals do the same job with tools such as spaCy, NLTK, scikit-learn and Hugging Face, so that when you use them you know exactly what they compute, and the last lesson connects it all to how large language models generate text and are used in practice.
What you will learn
- Clean, tokenize and normalize text, and extract patterns with regular expressions
- Represent documents with bag-of-words, n-grams and TF-IDF, and rank them for search
- Train and evaluate Naive Bayes and sentiment classifiers with precision, recall and F1
- Build smoothed n-gram language models and measure them with perplexity
- Tag parts of speech with an HMM and Viterbi, find named entities and correct spelling
- Use word embeddings, train a BPE tokenizer and compute attention and LLM decoding
Skills you will gain
- Text preprocessing and regular expressions
- Vector representations: counts, TF-IDF and embeddings
- Text classification and evaluation
- Language models, sequence labelling and dynamic programming
- Tokenizers, attention and LLM decoding
Before you start
- Python: strings, lists, dictionaries, loops and functions (the Python course)
- Basic probability: fractions, conditional probability and logarithms (the Statistics & Probability course helps)
- Comfort with simple algebra and vectors; everything else is explained when it is needed
Where it is used
- Spam filtering and sentiment analysis
- Search engines and document retrieval
- Spelling correction and autocomplete
- Information extraction from documents
- Chatbots and LLM applications
Modules
Module 1: Text basics
What NLP is and the text pipeline, tokenization and normalization, and regular expressions for finding and cleaning text.
- 1
What natural language processing is Intermediate
What NLP is, why human language is hard for computers, and the pipeline almost every NLP system follows: raw text, sentences, tokens, counts, features and a model. Build a tiny pipeline in plain Python.
- 2
Tokenization and normalization Intermediate
Split raw text into tokens and clean them up: separate punctuation, expand contractions, lowercase, remove stopwords and cut words to their stems. See what each step gains and what it can destroy, such as the word "not".
- 3
Regular expressions for text Intermediate
Find, extract and replace patterns in text with Python's re module: character classes, quantifiers, groups, anchors and alternation. Pull out emails, dates and hashtags, and write a regular-expression tokenizer.
Module 2: Counting words
Bag-of-words vectors and n-grams, TF-IDF weighting, and similarity, inverted indexes and ranking for search.
- 1
Bag of words and n-grams Intermediate
Turn documents into vectors of word counts over a shared vocabulary, the bag-of-words model, and recover some word order with n-grams. See what a bag keeps, what it throws away, and why the vocabulary grows so fast.
- 2
TF-IDF: weighting words Intermediate
Weight each word by how often it appears in a document (term frequency) and how rare it is across the collection (inverse document frequency). Compute TF-IDF by hand and in Python, and use it to find a document's keywords.
- 3
Text similarity and search Intermediate
Measure how alike two texts are with Jaccard and cosine similarity, find candidate documents fast with an inverted index, and rank them against a query with TF-IDF vectors: the classic design of a search engine.
Module 3: Classic models
Naive Bayes text classification, lexicon and classifier approaches to sentiment with proper evaluation, and n-gram language models with smoothing and perplexity.
- 1
Naive Bayes text classification Intermediate
Classify text with multinomial Naive Bayes: estimate class priors and word likelihoods from counts, smooth them with add-one, and add log probabilities to pick the most likely class. Build a spam filter from scratch.
- 2
Sentiment analysis Intermediate
Decide whether text is positive or negative: score words with a sentiment lexicon, handle negation and intensifiers, compare rules with trained classifiers, and measure the results with precision, recall and F1.
- 3
Language models and smoothing Intermediate
A language model gives every sentence a probability. Build bigram models from counts with sentence markers, see why unseen word pairs break maximum-likelihood estimates, fix them with add-one and add-k smoothing, measure models with perplexity and generate text.
Module 4: Structure in text
Part-of-speech tagging with hidden Markov models and the Viterbi algorithm, named entities with BIO tags and chunking, and edit distance for spelling correction.
- 1
Part-of-speech tagging with an HMM Intermediate
Label each word as a noun, verb, determiner and so on. Start from a most-frequent-tag baseline, estimate a hidden Markov model's transition and emission probabilities from a tagged corpus, and find the best tag sequence with the Viterbi algorithm.
- 2
Named entities and chunking Intermediate
Find the people, places, organizations and dates in text, encode them with BIO tags, read spans back out of tag sequences, chunk noun phrases from part-of-speech patterns, and score a recognizer at the entity level.
- 3
Edit distance and spelling correction Intermediate
Measure how far apart two strings are with Levenshtein edit distance, computed by dynamic programming; add adjacent swaps for typing errors; and build a spelling corrector that picks the most frequent dictionary word within a small distance.
Module 5: Meaning and modern NLP
Word embeddings, similarity and analogies, byte-pair-encoding tokenizers, and attention, transformers and how large language models generate text and are used.
- 1
Word embeddings Intermediate
Represent words as short dense vectors whose directions carry meaning. Compare words with cosine similarity, find nearest neighbours, build vectors from co-occurrence counts, and solve analogies such as king − man + woman with vector arithmetic.
- 2
Byte-pair encoding tokenization Intermediate
Learn how modern language models split text into sub-word tokens. Train byte-pair encoding by repeatedly merging the most frequent pair of symbols, then apply the learned merges to tokenize new words, and see how vocabulary size, unknown words and token counts trade off.
- 3
Attention, transformers and LLMs in practice Advanced
See how self-attention lets every token look at every other token, how transformers stack attention into large language models, and how an LLM turns scores into text with softmax, temperature, greedy, top-k and top-p decoding. Ends with how LLMs are used in practice.
Projects
- A product review analyser Build a complete text classification pipeline in plain Python that labels product reviews as positive or negative, explains its decisions and measures how well it works.
- A small search engine with spelling correction Build a search engine over a collection of short documents that corrects misspelled queries, ranks the results and shows the passage that matched.
What to learn next
- Deep Learning Neurons, backpropagation, optimizers, normalization, CNNs, RNNs, embeddings, attention and transformers, every network built from scratch in Python.
- Machine Learning Machine learning from scratch in plain Python: regression, gradient descent, regularization, k-NN, logistic regression, metrics, trees, ensembles and k-means.
- Artificial Intelligence Search and A*, game playing, constraint solving, logic, Bayesian networks, hidden Markov models, MDPs and Q-learning, each built in Python.