Embeddings, search and retrieval
Vectors that stand for meaning, semantic search over chunks, and retrieval-augmented generation with sources.
About 4 hours of study
Learning objectives
- Compute dot product and cosine similarity
- Chunk text and rank it for a query
- Build a grounded prompt with numbered sources
Lessons
- 1
Embeddings and similarity Intermediate
How models represent meaning as lists of numbers: embedding vectors, the dot product, cosine similarity, nearest neighbours and analogies by vector arithmetic.
- 2
Semantic search Intermediate
Finding text by meaning instead of exact words: splitting documents into chunks, embedding chunks and queries, ranking by cosine similarity, and comparing keyword and semantic search.
- 3
Retrieval-augmented generation (RAG) Intermediate
Giving a language model the right facts at the right time: retrieve relevant chunks, put them in the prompt with instructions to cite them, and generate an answer grounded in your own documents.
Module practice
- Dot product and cosine similarityEasy
- Nearest words by cosine similarityMedium
- Solve an analogy with vectorsHard
- Split a text into overlapping chunksEasy
- Rank documents by word-count similarityMedium
- Semantic search with averaged embeddingsHard
- Score documents by shared wordsEasy
- Build a grounded RAG promptMedium
- An extractive RAG answer with a citationHard
Module quiz
‹ How language models workBuilding with models and judging them ›