Building with models and judging them
Prompt templates, validated JSON and tool loops, and the scores used to evaluate generated text.
About 5 hours of study
Learning objectives
- Write prompts with templates and examples
- Validate structured output and run a tool-calling loop
- Compute exact match, n-gram scores and a BLEU-style score
Lessons
- 1
Prompt engineering Intermediate
Writing instructions a model can follow: the parts of a good prompt, zero-shot and few-shot prompting, templates with placeholders, system and user messages, and how to test a prompt.
- 2
Structured output and tool calling Intermediate
Making a model's answer usable by programs: asking for JSON, validating it, retrying on errors, and the tool-calling loop in which a model requests functions and your code runs them.
- 3
Evaluating generative models Intermediate
How to tell whether generated text is good: exact match, n-gram overlap scores such as BLEU and ROUGE, perplexity, and human or model-as-judge ratings, with the strengths and limits of each.
Module practice
- Fill a prompt templateEasy
- Build a few-shot promptMedium
- Pick few-shot examples within a word budgetHard
- Check a JSON replyEasy
- Retry until the reply is validMedium
- Run a tool-calling loopHard
- Exact match accuracyEasy
- N-gram precision and recallMedium
- A BLEU-style score with a brevity penaltyHard
Module quiz
‹ Embeddings, search and retrievalImages, tuning and safety ›