Images, tuning and safety
Diffusion on tiny images, fine-tuning and preference training, and the guards used for responsible use.
About 4 hours of study
Learning objectives
- Run a noise and denoise demonstration
- Count LoRA parameters and score preference pairs
- Redact personal data and flag injection and unsupported claims
Lessons
- 1
Image generation Intermediate
How machines make pictures: images as grids of numbers, diffusion models that learn to remove noise, and the ideas behind GANs and VAEs, shown on tiny one-dimensional images.
- 2
Fine-tuning and alignment Advanced
How a general model becomes a useful assistant: pretraining, fine-tuning on examples, preference data with RLHF and DPO as ideas, and LoRA, which trains only a tiny share of the weights.
- 3
Safety and responsible use Advanced
Using generative AI without harm: hallucination, prompt injection, privacy, bias and copyright, and simple checks in code that reduce each risk but never remove it.
Module practice
- Draw a pixel grid as textEasy
- Run the forward noising stepsMedium
- Denoise with a nearest-pattern denoiserHard
- Count LoRA parametersEasy
- Fine-tune a bigram modelMedium
- Score preference pairs with a reward modelHard
- Redact emails and phone numbersEasy
- Scan for prompt injectionMedium
- Flag unsupported claimsHard