A prompt, tool and evaluation test bench

The problem

Build a small test bench that fills prompt templates, runs a scripted model through a validated tool-calling loop, and scores the results, so you can see which prompt wording and which checks work best.

Requirements

Technologies

Step by step

  1. Write the template, the inputs and the right answers.
  2. Write the stand-in model so that its mistakes are easy to recognise.
  3. Write the validator, the retry loop and the tool loop.
  4. Score each prompt variant on the full set of inputs.
  5. Write a short note on which variant won and why you trust the result.

Expected result

A test bench that prints a score table for each prompt variant and a log of every rejected reply.

How it is evaluated

Ways to improve it