Evaluation Datasets: Test Cases, Edge Cases and Volume

1.From success criteria to a test case

An evaluation dataset exists to measure your success criteria, so the criteria decide what the dataset must contain. Anthropic's guidance notes that most applications are judged on several dimensions at once, such as accuracy, tone, privacy handling and latency. Each dimension needs test cases that can show it. A dataset built only for accuracy cannot tell you whether the model leaks private data.

Each entry in the dataset follows the same structure. The Anthropic cookbook lists four parts. First, an input prompt. This is often a set of variables that get dropped into a prompt template when the test runs. Second, the output the model produces for that input. Third, a golden answer to compare the output against. Fourth, a score produced by a grading method. The golden answer can take two forms. It can be a string the output must match exactly. Or it can be an example of an ideal answer that a grader compares the output against.

The docs' sentiment example shows the first form. Each test case pairs a tweet with a human-assigned label. That label is the golden answer, and a simple string comparison can score it. Look at the comments in the sample: the dataset author has marked which cases are edge cases.

tweets = [
    {"text": "This movie was a total waste of time. 👎", "sentiment": "negative"},
    {"text": "The new album is 🔥! Been on repeat all day.", "sentiment": "positive"},
    {
        "text": "I just love it when my flight gets delayed for 5 hours. #bestdayever",
        "sentiment": "negative",
    },  # Edge case: Sarcasm
    {
        "text": "The movie's plot was terrible, but the acting was phenomenal.",
        "sentiment": "mixed",
    },  # Edge case: Mixed sentiment
    # ... 996 more tweets
]

Not every golden answer is a single label. For an open-ended request, such as a workout plan with required numbers of reps, the cookbook writes the golden answer as a description of what a correct response must and must not include. A human or a model grader then checks the output against that description.

Sources12

2.Mirror real traffic, then add the edge cases

A dataset of tidy, typical inputs gives you a flattering score that does not hold up in production. The docs first ask for evals that match the real task distribution. They then name the edge cases worth adding: input data that is irrelevant or missing, input that is too long, poor or harmful user input in chat applications, and ambiguous cases where even humans would struggle to agree.

The example datasets in the docs apply this to each task. The table shows the edge cases the docs flag for each one.

Writing these cases has another benefit. Anthropic's agent-evals guidance points out that two engineers can read the same spec and handle edge cases differently. Writing the case down, with its expected result, settles which behaviour is correct. The same guidance says to take realistic tasks from the failures you actually see. Each production failure then becomes a new test case.

Sources13

3.Volume, automation and repeated trials

How large should the dataset be, and how carefully should each case be graded? The docs give two linked rules. First, automate grading when possible: write questions so a machine can grade them, for example as multiple-choice, string-match, code-graded or LLM-graded questions. Second, prefer volume over quality. Many questions graded automatically, each with slightly weaker signal, beat a few questions graded carefully by hand.

The cookbook explains why from cost. An eval has two expensive parts. Writing questions and golden answers is usually a one-off cost, and Claude can help by generating the questions. Grading is a cost you pay every time the eval runs, and you will run it often. So the dataset should be designed for grading that is fast and cheap.

The docs' example criterion also measures performance on a *held-out* test set of 10,000 diverse posts. Scoring on data kept apart from development is what makes the number credible.

Two more design choices come from Anthropic's agent-evals guidance. Model outputs vary between runs, so each task is run for several trials to get more consistent results. And a capability eval, which asks what the system can do well, should start with a low pass rate. A suite the system already passes gives you nothing to improve on.

Sources123

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.A small set of carefully hand-graded questions is better than a large set graded automatically.Why is that wrong?

   The docs say the opposite. Prefer volume, and design questions so they can be graded automatically.

   Covered in Volume, automation and repeated trials

2. 2.A good capability eval is one your system already mostly passes.Why is that wrong?

   Capability evals should start with a low pass rate. They should target tasks the system struggles with, so there is room to improve.

   Covered in Volume, automation and repeated trials