Natural Language Processing

Tokens, regular expressions, TF-IDF, search, Naive Bayes, sentiment, language models, HMM tagging, entities, spelling, embeddings, BPE and attention, all in plain Python.

Level
Intermediate
Time
About 22 hours
Modules
5
Lessons
15
Exercises
45

Natural language processing (NLP) lets computers read, sort, search and write human language: spam filters, search engines, translation, voice assistants and chatbots all depend on it. This course builds its core ideas step by step: cleaning and tokenizing text, regular expressions, bag-of-words and TF-IDF, similarity and search, Naive Bayes and sentiment analysis, n-gram language models, part-of-speech tagging with hidden Markov models, named entities, edit distance and spelling correction, word embeddings, byte-pair encoding and attention.

Everything is plain Python that runs in your browser, with no libraries to install, on texts small enough to check by hand. Each lesson also shows how professionals do the same job with tools such as spaCy, NLTK, scikit-learn and Hugging Face, so that when you use them you know exactly what they compute, and the last lesson connects it all to how large language models generate text and are used in practice.

What you will learn

  • Clean, tokenize and normalize text, and extract patterns with regular expressions
  • Represent documents with bag-of-words, n-grams and TF-IDF, and rank them for search
  • Train and evaluate Naive Bayes and sentiment classifiers with precision, recall and F1
  • Build smoothed n-gram language models and measure them with perplexity
  • Tag parts of speech with an HMM and Viterbi, find named entities and correct spelling
  • Use word embeddings, train a BPE tokenizer and compute attention and LLM decoding

Skills you will gain

  • Text preprocessing and regular expressions
  • Vector representations: counts, TF-IDF and embeddings
  • Text classification and evaluation
  • Language models, sequence labelling and dynamic programming
  • Tokenizers, attention and LLM decoding

Before you start

  • Python: strings, lists, dictionaries, loops and functions (the Python course)
  • Basic probability: fractions, conditional probability and logarithms (the Statistics & Probability course helps)
  • Comfort with simple algebra and vectors; everything else is explained when it is needed

Where it is used

  • Spam filtering and sentiment analysis
  • Search engines and document retrieval
  • Spelling correction and autocomplete
  • Information extraction from documents
  • Chatbots and LLM applications

Modules

Module 1: Text basics

What NLP is and the text pipeline, tokenization and normalization, and regular expressions for finding and cleaning text.

  1. 1

    What natural language processing is Intermediate

    What NLP is, why human language is hard for computers, and the pipeline almost every NLP system follows: raw text, sentences, tokens, counts, features and a model. Build a tiny pipeline in plain Python.

  2. 2

    Tokenization and normalization Intermediate

    Split raw text into tokens and clean them up: separate punctuation, expand contractions, lowercase, remove stopwords and cut words to their stems. See what each step gains and what it can destroy, such as the word "not".

  3. 3

    Regular expressions for text Intermediate

    Find, extract and replace patterns in text with Python's re module: character classes, quantifiers, groups, anchors and alternation. Pull out emails, dates and hashtags, and write a regular-expression tokenizer.

Module practice and quiz →

Module 2: Counting words

Bag-of-words vectors and n-grams, TF-IDF weighting, and similarity, inverted indexes and ranking for search.

  1. 1

    Bag of words and n-grams Intermediate

    Turn documents into vectors of word counts over a shared vocabulary, the bag-of-words model, and recover some word order with n-grams. See what a bag keeps, what it throws away, and why the vocabulary grows so fast.

  2. 2

    TF-IDF: weighting words Intermediate

    Weight each word by how often it appears in a document (term frequency) and how rare it is across the collection (inverse document frequency). Compute TF-IDF by hand and in Python, and use it to find a document's keywords.

  3. 3

    Text similarity and search Intermediate

    Measure how alike two texts are with Jaccard and cosine similarity, find candidate documents fast with an inverted index, and rank them against a query with TF-IDF vectors: the classic design of a search engine.

Module practice and quiz →

Module 3: Classic models

Naive Bayes text classification, lexicon and classifier approaches to sentiment with proper evaluation, and n-gram language models with smoothing and perplexity.

  1. 1

    Naive Bayes text classification Intermediate

    Classify text with multinomial Naive Bayes: estimate class priors and word likelihoods from counts, smooth them with add-one, and add log probabilities to pick the most likely class. Build a spam filter from scratch.

  2. 2

    Sentiment analysis Intermediate

    Decide whether text is positive or negative: score words with a sentiment lexicon, handle negation and intensifiers, compare rules with trained classifiers, and measure the results with precision, recall and F1.

  3. 3

    Language models and smoothing Intermediate

    A language model gives every sentence a probability. Build bigram models from counts with sentence markers, see why unseen word pairs break maximum-likelihood estimates, fix them with add-one and add-k smoothing, measure models with perplexity and generate text.

Module practice and quiz →

Module 4: Structure in text

Part-of-speech tagging with hidden Markov models and the Viterbi algorithm, named entities with BIO tags and chunking, and edit distance for spelling correction.

  1. 1

    Part-of-speech tagging with an HMM Intermediate

    Label each word as a noun, verb, determiner and so on. Start from a most-frequent-tag baseline, estimate a hidden Markov model's transition and emission probabilities from a tagged corpus, and find the best tag sequence with the Viterbi algorithm.

  2. 2

    Named entities and chunking Intermediate

    Find the people, places, organizations and dates in text, encode them with BIO tags, read spans back out of tag sequences, chunk noun phrases from part-of-speech patterns, and score a recognizer at the entity level.

  3. 3

    Edit distance and spelling correction Intermediate

    Measure how far apart two strings are with Levenshtein edit distance, computed by dynamic programming; add adjacent swaps for typing errors; and build a spelling corrector that picks the most frequent dictionary word within a small distance.

Module practice and quiz →

Module 5: Meaning and modern NLP

Word embeddings, similarity and analogies, byte-pair-encoding tokenizers, and attention, transformers and how large language models generate text and are used.

  1. 1

    Word embeddings Intermediate

    Represent words as short dense vectors whose directions carry meaning. Compare words with cosine similarity, find nearest neighbours, build vectors from co-occurrence counts, and solve analogies such as king − man + woman with vector arithmetic.

  2. 2

    Byte-pair encoding tokenization Intermediate

    Learn how modern language models split text into sub-word tokens. Train byte-pair encoding by repeatedly merging the most frequent pair of symbols, then apply the learned merges to tokenize new words, and see how vocabulary size, unknown words and token counts trade off.

  3. 3

    Attention, transformers and LLMs in practice Advanced

    See how self-attention lets every token look at every other token, how transformers stack attention into large language models, and how an LLM turns scores into text with softmax, temperature, greedy, top-k and top-p decoding. Ends with how LLMs are used in practice.

Module practice and quiz →

Projects

Final assessment →

What to learn next