Images, tuning and safety

Diffusion on tiny images, fine-tuning and preference training, and the guards used for responsible use.

About 4 hours of study

Learning objectives

Lessons

  1. 1

    Image generation Intermediate

    How machines make pictures: images as grids of numbers, diffusion models that learn to remove noise, and the ideas behind GANs and VAEs, shown on tiny one-dimensional images.

  2. 2

    Fine-tuning and alignment Advanced

    How a general model becomes a useful assistant: pretraining, fine-tuning on examples, preference data with RLHF and DPO as ideas, and LoRA, which trains only a tiny share of the weights.

  3. 3

    Safety and responsible use Advanced

    Using generative AI without harm: hallucination, prompt injection, privacy, bias and copyright, and simple checks in code that reduce each risk but never remove it.

Module practice

Module quiz

1. A diffusion model generates a picture by...
2. LoRA fine-tuning trains...
3. Prompt injection is...
4. LoRA of rank 4 on a layer with 1000 inputs and 500 outputs trains how many numbers?
5. A reward model gives the chosen and the rejected answer the same score. The sigmoid of the gap is...
6. A sentence has 50% of its words in the source and the threshold is 0.6. The check...

‹ Building with models and judging themFinal assessment ›