What you will be able to do
- Rewrite a vague quality goal as specific, measurable, multidimensional success criteria that can gate a release
- Select a defensible metric for each criterion and explain why it tracks the quality goal
- Assemble a golden test set that mirrors real traffic and its edge cases, and separate capability evals from regression evals
- Choose code-based, LLM-based or human grading for each criterion, including behavioral checks on tool calls and outcomes
- Design an LLM-as-judge rubric that produces reliable scores, and calibrate it against human-labeled data before trusting it
- Reuse a human-labeled golden set to measure a judge's agreement with human graders on cases it was not tuned on
Key concept
Success criterion as an eval target — An evaluation is only as good as the goal it measures. A usable success criterion names the quality that matters, attaches a number or a well-defined scale to it, and fixes the test set it is measured on, so each graded run gives a pass or fail you can act on.
1.Turning a quality goal into success criteria
"Summaries should be good" can't gate a release because nothing can fail it. A success criterion has to be specific about what quality is wanted, measurable with a number or a well-defined scale, achievable against a known baseline, and relevant to what the application is for. The sentiment example in Anthropic's guidance shows all four in one sentence: an F1 score of at least 0.85, on a held-out test set of 10,000 diverse Twitter posts, set as a 5% improvement over the current baseline.
Relevance is usually where a rewritten criterion goes wrong. Take a support-ticket summarizer. "Under 120 words with median generation time below 3 seconds" is specific and measurable, but it measures length and speed. A short, fast summary can still be wrong. When the real goal is a good summary, the criterion needs to measure what good means for that product, such as accuracy and completeness. One way to do that is a 1–5 rubric score with a threshold (at least 4 on 95% of a fixed ticket set). Qualitative scales are allowed: a Likert or expert rubric counts as measurable as long as it is applied consistently.
A metric can be easy to compute and still not match the goal. ROUGE-L is a recognised summary metric. It scores the longest common subsequence shared with a reference summary, so it rewards wording that overlaps one author's reference. A summary that gets every fact right in different words scores low. Requiring a ROUGE threshold on every single ticket makes that brittleness worse. Asking a product owner to approve 20 hand-picked summaries fails differently: it is relevant, but the sample is too small and the result can't be repeated automatically on every release.
A defensible metric is one you can justify from the quality goal, not from convenience. Start from the quality, then pick the measure whose failures look like the failures users would notice. Exact match suits categorical answers such as a sentiment label. Cosine similarity between embeddings of answers to paraphrased questions suits consistency, where wording may vary but meaning should not. A 1–5 ordinal scale with defined endpoints suits context use, which no string comparison can see. ROUGE-L suits summaries only when overlap with a reference is a fair stand-in for quality. Generic measures (accuracy, precision, recall), task-specific ones (F1, BLEU, perplexity) and operational ones (response time, uptime) each answer a different question, so a criterion usually needs one from more than one family. Finally, the pass line has to come from a baseline, prior experiments or expert knowledge, not from a number that merely sounds good.
| Dimension | Bad | Good |
|---|---|---|
| Task quality | The model should classify sentiments well | F1 score of at least 0.85 on a held-out set of 10,000 diverse Twitter posts |
| Safety | Safe outputs | 99.5% non-toxic outputs |
| Error severity | (unstated) | 90% of errors would cause inconvenience, not egregious error |
| Latency | (unstated) | 95% response time < 200ms |
Most applications need several criteria, each with its own threshold. Other dimensions worth considering are consistency across similar inputs, relevance and coherence, tone, privacy handling, context use, latency and cost. Keep them as separate pass/fail lines rather than one blended quality number. A blended score lets a release that is accurate but too slow average its way to a pass. Separate thresholds make the latency line fail on its own.
Checkpoint 1 of 8· Check yourself
A team needs a release gate for a support-ticket summarizer. The goal is summaries that support agents can trust. Which success criterion best fits that goal?
A criterion has to be relevant to what the application is for. The rubric option measures accuracy and completeness on a fixed set, and the run can be repeated. Length and speed don't measure whether a summary is good. ROUGE-L rewards word overlap with one reference summary. A 20-summary sign-off is too small to rely on and can't be rerun automatically on every release.
“Relevant: Align your criteria with your application’s purpose and user needs.”Source: platform.claude.com
Checkpoint 2 of 8· Exam question
A team ships a support-ticket summarizer built on Claude. Their only goal statement is "summaries should be good". They need something they can gate releases on. Which rewrite is the most usable success criterion?
Correct answer: B — Require a score of at least 4 out of 5 on an accuracy and completeness rubric for 95% of a fixed 300-ticket test set.
- A. Incorrect: word count and latency are measurable and repeatable on a fixed set, but they say nothing about whether a summary is correct or covers the ticket. A short, fast, wrong summary would pass, so the quality goal is left unmeasured.
- B. Correct: it defines the quality dimensions (accuracy and completeness), a scoring scale, a pass threshold and a fixed test set. That makes it specific, measurable and repeatable across releases, so it can serve as a release gate.
- C. Incorrect: a pass/fail from one person on a small random sample has no defined dimensions or threshold, and the sample changes each time. Results cannot be compared between releases, so it cannot gate them reliably.
- D. Incorrect: ROUGE-L against a single reference written by the prompt's author measures wording overlap, not accuracy or completeness. A correct summary phrased differently scores low, and a fluent but wrong one can score high.
2.Golden sets, edge cases and regression suites
Each test case pairs an input with a golden answer. The golden answer is either the exact output the model must produce, or a model answer that gives a grader something to compare against. Running the model on the input gives an output, and grading that output against the golden answer gives a score. Writing questions and golden answers is mostly a one-time cost. Grading is paid on every run, and a useful eval runs often, so cheap grading should drive the design.
The test set should look like real traffic. That means a similar mix of question types and difficulties, plus the edge cases that break naive prompts:
- irrelevant or nonexistent input data
- overly long input
- poor, harmful or irrelevant user input in chat applications
- ambiguous cases where even humans would struggle to agree
The sentiment dataset in the guidance labels some edge cases on purpose. A sarcastic tweet about a five-hour flight delay is labeled negative, and a review with terrible plot but phenomenal acting is labeled mixed. A consistency eval for an FAQ bot groups paraphrases of the same question, including one full of typos and one long, rambling version. If real inputs can't be used, for example for privacy reasons, Claude can generate synthetic values for a prompt template's variables.
Measure the criterion on a held-out test set: a fixed bank of cases kept apart from the examples you tune the prompt on. Because the bank is fixed, scores from different runs can be compared. Once it exists, it also gives you baselines for latency, token usage, cost per task and error rates at no extra effort.
Suites have two different jobs. Capability evals ask what the system can do well. They should start at a low pass rate so the team has room to improve. Regression evals ask whether the system still handles everything it used to, and should pass at nearly 100%. A drop there means a change broke something. When a capability task reaches a high pass rate, it can graduate into the regression suite and run continuously to catch drift.
The same human-labeled cases that grade a model can also be used to check an LLM judge. When the golden set carries human labels, such as 1,000 tweets with human-labeled sentiments, run the judge over the model outputs for a sample of those cases and compare its verdicts with the human ones. A judge that agrees with the labels on the cases you chose earns trust for that kind of case. Model-based graders need this comparison because their accuracy depends on calibration with human graders. Keep the labeled calibration cases apart from the examples used to tune the judge prompt, for the same reason the test set is held out: agreement measured on the cases you tuned against flatters the judge. Include the ambiguous cases, where humans themselves disagree, and read them separately from clear-cut ones. The rubric design and calibration steps later on this page cover the procedure.
Checkpoint 3 of 8· Match them up
A team is setting up the test suites for a Claude feature. Match each eval component to the job it does.
Tap a term, then the definition that fits it.
Capability and regression suites do opposite jobs. A capability eval measures what the system can't do well yet, so a low pass rate is expected. A regression eval checks that nothing has broken, so it should pass at nearly 100%. A held-out set is fixed, which makes runs comparable, and the golden answer is what each output is graded against.
“Regression evals ask, “Does the agent still handle all the tasks it used to?” and should have a nearly 100% pass rate.”Source: www.anthropic.com
3.Choosing code, model or human grading
Pick the fastest grading method that is reliable for the criterion. Code-based grading (exact match, string or regex checks, static analysis) is fast, cheap and reproducible, but it can't make nuanced judgments. Human grading handles almost any task, but it is slow and expensive, so avoid it where you can. LLM-based grading sits in between. It is flexible enough for open-ended output and scalable, but you have to test it before relying on it.
| Method | Strengths | Weaknesses | Wins when |
|---|---|---|---|
| Code-based | Fast, cheap, objective, reproducible, easy to debug | Brittle to valid variations; lacks nuance | Answers are categorical or checkable: a label, a number, passing tests |
| LLM-based (judge) | Flexible, scalable, captures nuance, handles freeform output | Non-deterministic; costs more than code; needs calibration with human graders | Open-ended answers judged against a rubric |
| Human | Gold standard quality; matches expert judgment | Expensive, slow, needs experts at scale | Labeling calibration data and spot-checking samples |
A careful task design often makes code grading possible. If the sentiment prompt asks for "Output only the label", one line of normalized comparison grades 1,000 tweets. Turning free-form questions into multiple choice is another common way to make grading automatable. When the budget is tight, choose volume: many automatically graded questions with slightly lower signal beat a few hand-graded ones.
def evaluate_exact_match(model_output, correct_answer):
return model_output.strip().lower() == correct_answer.lower()Behavioral checks grade what the system did, not only what it said. Code graders can check which tools were called and with which parameters, or count turns and tokens in the transcript. For agents, grade the outcome as well as the transcript. A flight-booking agent might say the flight is booked, but the outcome is whether a reservation exists in the database. Some behaviors are about refusing correctly. One golden answer requires the assistant to decline to send an email because it has no way to send email: a draft is fine, but calling a send function fails the case. Several graders can score one task, either as a weighted total that must reach a threshold or as a set that must all pass.
Checkpoint 4 of 8· Check yourself
Your support assistant writes free-form answers, and each one has to be graded against a rubric on every release. The budget for grading is tight. Which grading plan is easiest to defend?
Open-ended answers judged against a rubric are where an LLM judge does best. A judge has to be calibrated against human graders before you rely on it. Three runs that agree only show the judge is consistent with itself, not that it agrees with human judgment. Exact match is too brittle for answers that can be worded in many valid ways. A few hand-graded cases lose to many automatically graded ones when the budget is tight.
“Requires calibration with human graders for accuracy”Source: www.anthropic.com
4.Designing a reliable LLM judge
An LLM judge scores consistently only when it knows exactly what each score means. The guidance gives three rules. First, write detailed, clear rubrics with checkable conditions. Its example says an answer that doesn't mention "Acme Inc." in the first sentence is automatically incorrect. Second, make the output empirical or specific: only "correct" or "incorrect", or a defined 1–5 scale, because purely qualitative verdicts can't be assessed quickly or at scale. Third, encourage reasoning by running the grader with thinking on, so it reasons before it scores, which helps most on complex judgments.
def build_grader_prompt(answer, rubric):
return f"""Grade this answer based on the rubric:
<rubric>{rubric}</rubric>
<answer>{answer}</answer>
Output 'correct' or 'incorrect' in <result> tags."""Applied to a vague "quality 1–10" prompt, those rules mean splitting quality into named dimensions such as accuracy, completeness and tone. Each dimension gets written criteria and its own label, and the judge reasons before giving each label. Telling the judge to "be extremely strict" changes how harsh it is without saying what it is judging. A 1–100 scale adds resolution that no criterion defines. Averaging many high-temperature runs smooths the variation without making the judgment any better defined, and model-based graders are already non-deterministic. Score anchors help ordinal scales too: the context-use judge defines 1 as "Completely ignores context" and 5 as "Perfectly utilizes context".
To reduce grader bias, it is generally best practice to grade with a different model from the one that produced the output. Specialized judgments follow the same pattern. A judge that checks whether answers are supported by retrieved passages is a rubric judge whose criterion is grounding in source passages.
Checkpoint 5 of 8· Fill the gap
This grader prompt has to return a verdict that code can parse quickly and at scale. Which value completes the output instruction?
def build_grader_prompt(answer, rubric):
return f"""Grade this answer based on the rubric:
<rubric>{rubric}</rubric>
<answer>{answer}</answer>
Output 'correct' or ? in <result> tags."""A reliable judge gives an empirical or specific output, such as only 'correct' or 'incorrect', so a simple string check can parse the verdict. A 1–100 score adds detail that no criterion defines. Free-text feedback is hard to assess quickly or at scale. A judge's confidence in its own answer doesn't make the judgment any better defined.
Source: platform.claude.comCheckpoint 6 of 8· Exam question
A judge prompt asks Claude to "rate response quality from 1 to 10". Scores for the same response vary widely between runs and reviewers cannot explain them. Which redesign most improves reliability?
Correct answer: D — Split quality into named dimensions with explicit criteria, request one label per dimension, and let the judge reason before it.
- A. Incorrect: strictness changes the average but not the lack of defined criteria, so the scores stay unexplained.
- B. Incorrect: averaging many high-temperature runs costs more and hides the bias in the vague rubric rather than removing it.
- C. Incorrect: a finer scale adds false precision, and vague criteria produce even more run-to-run scatter.
- D. Correct: specific rubric dimensions with a small set of labels, plus reasoning before the verdict, make judgments more consistent and the results explainable.
Sources1
5.Calibrating the judge against human labels
A judge prompt is a measuring instrument, and an instrument nobody has checked can't gate a release. Model-based graders need calibration with human graders before you can trust their accuracy. Human labels are the reference they are calibrated against. The guidance puts it as test for reliability first, then scale up.
Calibrating a judge has three steps:
- Run the judge on responses that trained human reviewers have already labeled.
- Measure how often the judge's verdicts agree with the human labels.
- Read the mismatches, find where the rubric is ambiguous or wrong, revise it, and measure again.
This is the empirical test the guidance asks for: the only way to know whether a model-based grader can handle your task is to try it and read samples. Where human reviewers themselves disagree, inter-annotator agreement is the method for measuring that, and those ambiguous cases deserve a closer look before you blame the judge.
Several checks look like validation but never compare the judge with human judgment. The judge's own stated confidence is still the judge's opinion. Averaging several high-temperature verdicts per response reduces noise but not bias. A neatly shaped score distribution on unlabeled traffic shows how the scores are spread, not whether they are correct. Only agreement with human labels shows that.
Calibration also goes out of date. Teams that run LLM graders in production pair them with periodic human calibration. Descript, for example, moved from manual grading to LLM graders with product-defined criteria and periodic human calibration. If the judge prompt is edited, or traffic shifts toward new product areas the original labels didn't cover, label a fresh sample that reflects current traffic and measure agreement again before trusting the scores.
Checkpoint 7 of 8· Check yourself
Six months ago a team calibrated its LLM judge at 92% agreement with human labels. Since then they have rewritten the judge prompt, and the product now handles billing questions that the original labels never covered. What should happen before the judge's scores gate the next release?
Calibration goes out of date. The old figure measured a different prompt on different traffic, so agreement has to be measured again against fresh human labels. The shape of a score distribution and the judge's own confidence never compare the judge with human judgment.
“They evolved from manual grading to LLM graders with criteria defined by the product team and periodic human calibration”Source: www.anthropic.com
Checkpoint 8 of 8· Exam question
A team built an LLM judge to score chatbot helpfulness and has 200 responses labeled by trained human reviewers. Before using the judge to gate releases, what should they do first?
Correct answer: C — Run the judge on the 200 reviewed responses, measure its agreement with the reviewers' labels, and study mismatches to refine the rubric.
- A. Incorrect: the judge's self-reported confidence is not calibrated evidence of correctness, and nothing here compares verdicts with the reviewers' labels. Filtering on confidence can also discard exactly the cases where the judge is weak.
- B. Incorrect: averaging repeated verdicts reduces random variance but not systematic bias, and the mean is never compared with the reviewers' labels. A judge that diverges from human judgment stays wrong, just more consistently.
- C. Correct: the labeled responses let the team check whether the judge's verdicts track trained human judgment before it gates releases. Examining the mismatches shows which rubric wording to fix.
- D. Incorrect: the shape of a score distribution on unlabeled traffic says nothing about whether individual verdicts match what reviewers would decide. This ignores the labeled data that could validate the judge.
Sources
(4)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“Use quantitative metrics or well-defined qualitative scales.”
↩︎ Turning a quality goal into success criteria“achieve an F1 score of at least 0.85 (Measurable, Specific) on a held-out test set”
↩︎ Turning a quality goal into success criteria“It measures the length of the longest common subsequence between the candidate and reference summaries.”
↩︎ Turning a quality goal into success criteria“Task-specific: F1 score, BLEU score, perplexity Generic: Accuracy, precision, recall Operational: Response time (ms), uptime (%)”
↩︎ Turning a quality goal into success criteria“perfect for tasks with clear-cut, categorical answers such as sentiment analysis”
↩︎ Turning a quality goal into success criteria“similar questions should yield semantically similar answers, even if the wording varies”
↩︎ Turning a quality goal into success criteria“Most use cases need multidimensional evaluation along several success criteria.”
↩︎ Turning a quality goal into success criteria“Design evals that mirror your real-world task distribution.”
↩︎ Golden sets, edge cases and regression suites“Ambiguous test cases where even humans would find it hard to reach an assessment consensus”
↩︎ Golden sets, edge cases and regression suites“Example eval test cases: 1,000 tweets with human-labeled sentiments.”
↩︎ Golden sets, edge cases and regression suites“Fastest and most reliable, extremely scalable, but also lacks nuance for more complex judgments that require less rule-based rigidity.”
↩︎ Choosing code, model or human grading“Most flexible and high quality, but slow and expensive. Avoid if possible.”
↩︎ Choosing code, model or human grading“Purely qualitative evaluations are hard to assess quickly and at scale.”
↩︎ Designing a reliable LLM judge“Use a grader model with thinking on, so that it reasons before it produces an evaluation score.”
↩︎ Designing a reliable LLM judge“Generally best practice to evaluate with a different model from the one used to generate the evaluated output”
↩︎ Designing a reliable LLM judge“Test to ensure reliability first, then scale.”
↩︎ Calibrating the judge against human labels“Building a successful LLM-based application starts with clearly defining your success criteria and then designing evaluations to measure performance against them.”
↩︎ Key concept“Most use cases need multidimensional evaluation along several success criteria.”
↩︎ Exam trap 1“More questions with slightly lower signal automated grading is better than fewer questions with high-quality human hand-graded evals.”
↩︎ Exam trap 2“Relevant: Align your criteria with your application’s purpose and user needs.”
↩︎ Checkpoint“Have detailed, clear rubrics:”
↩︎ Prediction - 2.
“Brittle to valid variations”
↩︎ Turning a quality goal into success criteria“to become a regression suite that is run continuously to catch any drift.”
↩︎ Golden sets, edge cases and regression suites“Requires calibration with human graders for accuracy”
↩︎ Golden sets, edge cases and regression suites“The outcome is the final state in the environment at the end of the trial.”
↩︎ Choosing code, model or human grading“Used to calibrate model-based graders”
↩︎ Calibrating the judge against human labels“LLM graders with criteria defined by the product team and periodic human calibration”
↩︎ Calibrating the judge against human labels“should have a nearly 100% pass rate”
↩︎ Exam trap 3“Requires calibration with human graders for accuracy”
↩︎ Exam trap 4“Regression evals ask, “Does the agent still handle all the tasks it used to?” and should have a nearly 100% pass rate.”
↩︎ Checkpoint“Requires calibration with human graders for accuracy”
↩︎ Checkpoint“They evolved from manual grading to LLM graders with criteria defined by the product team and periodic human calibration”
↩︎ Checkpoint - 3.https://platform.claude.com/cookbook/misc-building-evalsSecondary source
“Grading on the other hand is a cost you will incur every time you re-run your eval, in perpetuity”
↩︎ Golden sets, edge cases and regression suites“A correct answer should decline to send the email since the assistant has no capabilities to send emails.”
↩︎ Choosing code, model or human grading“The only way to know if a model-based grader can do a good job grading your task is to try.”
↩︎ Calibrating the judge against human labels - 4.https://platform.claude.com/cookbook/misc-generate-test-casesSecondary source
“maybe you aren't allowed to test on the ones you do have for privacy reasons. No worries -- Claude can make them up!”
↩︎ Golden sets, edge cases and regression suites