Embeddings, search and retrieval

Vectors that stand for meaning, semantic search over chunks, and retrieval-augmented generation with sources.

About 4 hours of study

Learning objectives

Lessons

  1. 1

    Embeddings and similarity Intermediate

    How models represent meaning as lists of numbers: embedding vectors, the dot product, cosine similarity, nearest neighbours and analogies by vector arithmetic.

  2. 2

    Semantic search Intermediate

    Finding text by meaning instead of exact words: splitting documents into chunks, embedding chunks and queries, ranking by cosine similarity, and comparing keyword and semantic search.

  3. 3

    Retrieval-augmented generation (RAG) Intermediate

    Giving a language model the right facts at the right time: retrieve relevant chunks, put them in the prompt with instructions to cite them, and generate an answer grounded in your own documents.

Module practice

Module quiz

1. The cosine similarity of two vectors pointing the same way is...
2. In semantic search, a chunk is...
3. RAG stands for...
4. What is the dot product of [1, 2, 3] and [4, 5, 6]?
5. A RAG answer is wrong because the correct passage was never retrieved. What should be fixed first?
6. Why are chunks made to overlap?

‹ How language models workBuilding with models and judging them ›