What you will be able to do
- Use disagreement with human labels to tell that an LLM judge's scores are biased
- Name the bias behind a judge–human mismatch from its symptom: position, verbosity, self-preference or judge drift
- Pick the correction that fixes the bias actually present, and explain why fixes aimed at a different bias do not help
Key concept
Judge calibration against human labels — A model grader is only trustworthy for a task once its verdicts have been shown to agree with expert human grading on that task. When the judge and the humans disagree, the disagreement is the evidence of bias, and the pattern of the disagreement shows which bias it is.
1.Human labels are the reference a judge is measured against
An LLM judge is a model that grades another model's output against a rubric. It earns its place because it can do what code cannot: grade free-form answers that have no single correct string, and do it fast and at volume. Human grading is the most flexible and highest-quality option, but it is slow and expensive. Code-based grading is fast and reliable, but too rigid for nuanced judgements. LLM grading sits between the two, and it carries one condition: test that it is reliable first, then scale it up.
Reliable means that the judge agrees with expert humans on your task. Calibration means grading a sample with both the judge and people, then checking how far the two diverge. When they diverge, do not average the gap away, and do not trust whichever number looks better. A gap that leans consistently one way, such as toward the longer answer, the first-listed answer or the answer from one model family, is the fingerprint of a specific bias. Each bias has its own correction.
Reading what the judge actually wrote is how you identify which fingerprint you have. When a task fails, the transcript shows whether the output was really wrong or whether the grader rejected a valid answer. The same applies to a judge's per-answer rationales. If the rationales keep praising structure and phrasing while humans are scoring accuracy, the judge is grading something other than what the rubric meant. Scores are not taken at face value until someone has read the details behind them.
def build_grader_prompt(answer, rubric):
return f"""Grade this answer based on the rubric:
<rubric>{rubric}</rubric>
<answer>{answer}</answer>
Output 'correct' or 'incorrect' in <result> tags."""| Bias | Symptom against human review | Matched correction | Tempting fix that misses |
|---|---|---|---|
| Position | Judge favours whichever answer sits in one slot; swapping the order flips verdicts | Judge both orders; count a win only when both verdicts agree, otherwise a tie | Bigger judge model, temperature zero, or moving the challenger to the favoured slot |
| Verbosity | Scores rise with length while humans rate the answers equal | Split the rubric into specific criteria, give padding no credit, and control for length | Averaging several judge runs on the same vague rubric |
| Self-preference | Judge ranks its own model family's output first while blind humans disagree; rationales praise that family's style | Judge from a different family or a mixed panel, spot-checked against blind human labels | Asking the same judge to reason first or to check against reference answers |
| Judge drift | The same kind of output gets different scores across runs or iterations, while humans see no change | Anchor the scale with few-shot examples and detailed score breakdowns, and re-check against humans periodically | Trusting the trend line without reading samples |
Checkpoint 1 of 8· Check yourself
A team wants to replace slow human review of free-form support answers with an LLM judge. Before they run the judge across the full eval set, what establishes that its verdicts can be relied on?
A judge counts as reliable only once its verdicts are shown to agree with expert human grading on the task. Averaging runs and making the judge deterministic make it more consistent, but a consistent judge can still be biased. Exact matching is too rigid for free-form answers.
“LLM-as-judge graders should be closely calibrated with human experts to gain confidence that there is little divergence between the human grading and model grading.”Source: www.anthropic.com
2.Position bias: when the verdict follows the slot
A pairwise judge sees two candidates and picks one. If it leans toward whichever candidate fills a particular slot, its verdicts measure the presentation order instead of quality. The giveaway is a controlled swap. Re-run a sample with the two answers exchanged and nothing else changed. A judge reading content gives the same winner both times. A judge with position bias changes its answer.
Here is a concrete case. A team always lists the production summary first and the challenger second. Human reviewers prefer the challenger 60% of the time, but the judge picks the first-listed summary in 78% of trials, and swapping many pairs flips the verdict. A judge whose answer changes when only the order changes fails the most basic test of a grader, which is reliability.
The correction has to cancel the slot rather than hide it. Judge every pair twice, once in each order. Credit a win only when both verdicts name the same summary. When the verdict follows the slot, with A winning in one order and B winning in the other, the judge has no real preference, so score the pair as a tie. Each candidate then gets equal exposure to the favoured position, and only preferences that survive the swap count.
The tempting alternatives each leave the bias in place:
- Larger model tier, same order: the production summary still holds the favoured slot, so a lean toward that slot still inflates its wins.
- Temperature zero, same order: this makes the verdict repeatable, not correct. You get the same slot-driven answer every time.
- Challenger first for the whole run: the bias now favours the challenger, and comparing that win rate with the humans' 60% measures the slot again.
Checkpoint 2 of 8· Check yourself
In the summary comparison, the judge picks the first-listed production summary in 78% of trials, and swapping the order flips many verdicts. Human reviewers prefer the challenger 60% of the time. Which change gives a win rate that is worth comparing with the human labels?
Position bias follows the slot, so the fix has to give both candidates equal time in the favoured slot. Only preferences that hold after the swap should count, and the result is then checked against human labels. A larger model, temperature zero or averaging repeats the same slot-driven verdict. Putting the challenger first just moves the bias onto the other candidate.
“LLM-as-judge graders should be closely calibrated with human experts to gain confidence that there is little divergence between the human grading and model grading.”Source: www.anthropic.com
Checkpoint 3 of 8· Exam question
A team uses Claude through the Messages API as a pairwise judge to choose between two candidate summaries, always presenting the current production summary first. Human reviewers prefer the challenger in 60% of cases, but the judge picks the first-listed summary in 78% of trials. Re-running a sample with the two summaries swapped flips many verdicts. Which correction addresses the responsible bias?
Correct answer: A — Judge each pair in both orders, crediting a win only when the two verdicts agree and scoring pairs where the verdict follows the slot as ties.
- A. Correct. Verdicts that flip when the summaries are swapped are the signature of position bias. Judging both orders and crediting a winner only when the verdicts agree cancels the slot preference, and pairs where the verdict follows the slot are scored as ties.
- B. Incorrect. A larger tier may weaken the preference for the first slot, but it does not remove it. With the production summary still listed first, whatever bias remains skews each comparison. The fix has to change how pairs are presented.
- C. Incorrect. Listing the challenger first only moves the favored slot to the other summary, so the challenger's win rate is inflated instead. Each pair is still judged in a single order, so a verdict cannot be separated from its position.
- D. Incorrect. Temperature zero makes the judge more repeatable, but a deterministic bias is still a bias. The judge will keep favoring the first slot, so with the production summary listed first the skew stays. Repeatability does not make the verdicts valid.
3.Verbosity bias: when longer reads as better
Verbosity bias is a judge rewarding length for its own sake. It thrives on vague rubrics. Suppose a single 1–5 "helpfulness" score gives variant B 4.6 and variant A 3.9, while human reviewers rate the two as equal on accuracy and resolution. B's answers are three times longer, and the extra length is restated context and caveats. An open-ended question like "how helpful is this?" leaves the judge free to treat thoroughness-looking text as help, and purely qualitative grading is hard to do consistently at scale.
The first correction is the rubric. A good rubric is detailed and specific, down to rules that settle a verdict automatically, for example: an answer that omits a required item is graded incorrect. Replace "helpfulness" with the criteria humans actually used, accuracy and issue resolution, and state outright that padding and restated context earn no points. Grading each dimension against its own clear criteria also stops a strong impression on one dimension from leaking into the others. Concrete principles give the judge something concrete to grade against.
The second correction is a measurement check. Compare answers that have been trimmed to matched length, or add token count as a covariate, and test whether the judge's scores still track length. If the gap between A and B disappears once length is controlled, length was driving the score. If a gap remains, it may be real quality.
Averaging removes run-to-run noise, but verbosity bias is systematic. The judge gives B the length bonus on every run, so the average reproduces it more precisely. Randomizing answer order would not help either, because that targets position bias, and this scenario has no pairwise slot to randomize.
Checkpoint 4 of 8· Check yourself
Variant B's answers are three times longer than variant A's, and the extra length is mostly restated context. A single 1–5 "helpfulness" judge scores B at 4.6 and A at 3.9, but human reviewers rate the two as equal on accuracy and resolution. Which change targets the cause of the gap?
The gap comes from verbosity bias, which a vague rubric makes possible. Specific criteria that match what humans scored, with no credit for padding, remove the length bonus. Averaging only repeats the same systematic bias, randomizing order targets position bias, and changing the judge's model family targets self-preference.
“It can also help to create clear, structured rubrics to grade each dimension of a task”Source: www.anthropic.com
Checkpoint 5 of 8· Exam question
An internal eval scores customer-support answers from two prompt variants with a Claude judge and a 1–5 helpfulness rubric. Variant B averages 4.6 against 3.9 for variant A, yet human reviewers rate them equal on accuracy and resolution. B answers average three times the token length of A and mostly add restated context and caveats. Which two changes best address the responsible bias? Choose two.(Select 2)
Correct answers: A, B — Split the rubric into separate accuracy and issue-resolution criteria, and tell the judge that padding and restated context earn no points.; Compare answers after trimming them to matched length or adding token count as a covariate, then check whether judge scores still track length.
- A. Correct. A rubric with separate criteria for accuracy and resolution, plus an instruction that padding and restated context earn no points, steers the judge toward what the human reviewers measured. It directly reduces the judge's reward for longer answers.
- B. Correct. Matching length by trimming, or modelling token count as a covariate, separates the effect of length from the effect of the prompt. Checking whether scores still follow length shows whether the 4.6 vs 3.9 gap was a verbosity artifact.
- C. Incorrect. Averaging five runs reduces random run-to-run variance in the ratings. The judge's systematic preference for longer answers is the same in every run, so the averaged scores would still favor variant B.
- D. Incorrect. Randomizing presentation order counters position bias. The gap here follows answer length, and the original scores are absolute ratings, so there is no ordering effect to cancel. Variant B's longer answers would still win under both orderings.
- E. Incorrect. Self-preference is a judge favoring text from its own model family, and nothing in the scenario points to that. The gap tracks length, and a judge from another vendor can reward longer answers just as readily.
4.Self-preference: when the judge is generous to its own kind
Models grading their own output tend to praise it confidently, even when a human can see the work is mediocre, and the skew is strongest on subjective tasks. Separation helps, but it does not remove the leniency entirely: a standalone evaluator is still an LLM inclined to be generous toward LLM-generated outputs. Tuning a separate evaluator to be skeptical is far more tractable than making a generator critical of its own work.
Self-preference applies directly when the judge comes from the same family as one of the contestants. Take a benchmark comparing a Claude-based assistant with another vendor's assistant, judged by a Claude model. The judge ranks the Claude assistant first. A blind human panel ranks the other assistant first by a clear margin. The judge's rationales praise phrasing and structure typical of Claude output. All three signals point the same way: the judge is rewarding output that resembles its own.
The correction removes the shared identity. Use a judge from a different model family, or a mixed-family panel, and spot-check its ranking against blind human labels, because human review is the high-quality reference even when it is too slow to grade everything. The alternatives leave the same judge in charge:
- Reasoning before the score, with quoted passages: this makes the rationale longer, but the same preference still produces it.
- Reference answers and claim-checking: this improves factual grading, but the human panel's disagreement was not about facts the judge got wrong.
- Randomized order: this targets position bias, and nothing in this scenario points to the slot.
Checkpoint 6 of 8· Check yourself
Blind human reviewers rank a rival vendor's assistant clearly ahead of a Claude-based assistant. The benchmark's Claude judge ranks them the other way, and its rationales keep praising phrasing and structure typical of Claude output. Which change addresses what is skewing this judge?
The symptoms point to self-preference: the judge favours output that resembles its own family's. The skew comes from who is judging, so the fix is a different judge, checked against blind human labels. Reasoning first and reference answers leave the same lenient judge in charge, and randomizing order targets position bias, which nothing in this scenario suggests.
“Separating the agent doing the work from the agent judging it proves to be a strong lever to address this issue.”Source: www.anthropic.com
Checkpoint 7 of 8· Exam question
A company compares a Claude-based assistant against an assistant from another vendor on an internal knowledge-QA benchmark. A Claude model is the judge, and it ranks the Claude assistant first. A blind human panel ranks the other assistant first by a clear margin, and the judge's per-answer rationales praise phrasing and structure typical of Claude output. Which correction addresses the responsible bias?
Correct answer: A — Replace the Claude judge with one from a different model family, or a mixed-family panel, and spot-check its ranking against blind human labels.
- A. Correct. The judge favors phrasing and structure that resemble its own output, which is self-preference bias. A judge from another family, or a mixed panel, does not share the Claude assistant's style, and a sample of blind human labels confirms the ranking now matches human judgment.
- B. Incorrect. Reasoning before the score and quoted passages improve consistency and traceability, but the same Claude judge still finds its own style more convincing. It would cite Claude-typical phrasing as evidence, so the disagreement with the human panel remains.
- C. Incorrect. Randomizing answer order cancels position bias. The rationales point to familiar phrasing and structure rather than to which answer was shown first, so pooling shuffled verdicts from the same judge leaves the stylistic preference in place.
- D. Incorrect. Documentation and reference answers can improve factual grading, but they do not remove the judge's preference for answers written in its own style. Claude-style phrasing would still be rated more favorably, and the gap with the human panel would persist.
5.Judge drift: keeping the scale where you set it
Judge drift is the judge's scale moving over time. Equivalent work receives different scores across runs or iterations, and the trend line starts reflecting the grader rather than the system being graded. It is easy to miss, because a drifting judge still produces plausible-looking numbers. It shows up when you compare against fresh human labels or read samples from different points in time.
Anchoring the scale is the correction. Anthropic's write-up of a frontend-design harness shows the setting: a generator agent built a page, an evaluator agent scored it against the criteria, and that critique went back to the generator for the next round, with 5 to 15 of these generate-and-score iterations per run. The author calibrated the evaluator with few-shot examples that carried detailed score breakdowns, which aligned its judgment with their preferences and reduced score drift across those iterations. Worked examples pin each score level to concrete output, so the judge has a fixed reference instead of re-deciding what a 3 means on each run. Ask for empirical, specific output, such as a fixed label or a 1–5 scale with defined anchors, rather than open-ended commentary, so that a 3 means the same thing in week ten as it did in week one.
Calibration is an ongoing practice, not a one-time gate. Once a judge has proven robust against human grading, occasional human review is enough to keep it honest, but occasional is not never. Re-run the human comparison when anything about the judge changes, including its prompt, its rubric wording or the model behind it. Criterion wording alone can shift outputs in ways the designer did not anticipate, so a reworded rubric is effectively a new judge until it has been re-checked.
Checkpoint 8 of 8· Check yourself
A support-answer judge was checked against expert human grading in week one. Since then its rubric wording has been revised twice. By week ten its 1–5 scores are up 12%, but humans reading samples from both weeks see no change in quality. What is the right response?
Equivalent work scoring higher over time while humans see no change is judge drift. Few-shot anchors with detailed score breakdowns keep a score meaning the same thing from week to week. A reworded rubric is effectively a new judge, so it must be checked against humans again. Averaging only reduces noise, and changing the model family is the fix for self-preference.
“using few-shot examples with detailed score breakdowns. This ensured the evaluator’s judgment aligned with my preferences, and reduced score drift across iterations.”Source: www.anthropic.com
Sources
(4)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“Test to ensure reliability first, then scale.”
↩︎ Human labels are the reference a judge is measured against“Fast and flexible, scalable and suitable for complex judgment.”
↩︎ Human labels are the reference a judge is measured against“choose the fastest, most reliable, most scalable method”
↩︎ Position bias: when the verdict follows the slot“Have detailed, clear rubrics”
↩︎ Verbosity bias: when longer reads as better“Purely qualitative evaluations are hard to assess quickly and at scale.”
↩︎ Verbosity bias: when longer reads as better“Most flexible and high quality, but slow and expensive.”
↩︎ Self-preference: when the judge is generous to its own kind“Be empirical or specific”
↩︎ Judge drift: keeping the scale where you set it“choose the fastest, most reliable, most scalable method”
↩︎ Exam trap 2 - 2.
“When a task fails, the transcript tells you whether the agent made a genuine mistake or whether your graders rejected a valid solution.”
↩︎ Human labels are the reference a judge is measured against“we do not take eval scores at face value until someone digs into the details of the eval”
↩︎ Human labels are the reference a judge is measured against“gain confidence that there is little divergence between the human grading and model grading”
↩︎ Position bias: when the verdict follows the slot“grade each dimension with an isolated LLM-as-judge rather than using one to grade all dimensions”
↩︎ Verbosity bias: when longer reads as better“LLM-as-judge graders should be closely calibrated with human experts”
↩︎ Key concept“gain confidence that there is little divergence between the human grading and model grading”
↩︎ Exam trap 1“LLM-as-judge graders should be closely calibrated with human experts to gain confidence that there is little divergence between the human grading and model grading.”
↩︎ Checkpoint“LLM-as-judge graders should be closely calibrated with human experts to gain confidence that there is little divergence between the human grading and model grading.”
↩︎ Checkpoint“It can also help to create clear, structured rubrics to grade each dimension of a task”
↩︎ Checkpoint - 3.
“was the most consistent and aligned with human judgements”
↩︎ Verbosity bias: when longer reads as better - 4.
“gives Claude something concrete to grade against”
↩︎ Verbosity bias: when longer reads as better“agents reliably skew positive when grading their own work”
↩︎ Self-preference: when the judge is generous to its own kind“tuning a standalone evaluator to be skeptical turns out to be far more tractable than making a generator critical of its own work”
↩︎ Self-preference: when the judge is generous to its own kind“I calibrated the evaluator using few-shot examples with detailed score breakdowns.”
↩︎ Judge drift: keeping the scale where you set it“reduced score drift across iterations”
↩︎ Judge drift: keeping the scale where you set it“I ran 5 to 15 iterations per generation”
↩︎ Judge drift: keeping the scale where you set it“That feedback flowed back to the generator as input for the next iteration.”
↩︎ Judge drift: keeping the scale where you set it“The wording of the criteria steered the generator in ways I didn”
↩︎ Judge drift: keeping the scale where you set it“the evaluator is still an LLM that is inclined to be generous towards LLM-generated outputs”
↩︎ Exam trap 3“reduced score drift across iterations”
↩︎ Exam trap 4“Separating the agent doing the work from the agent judging it proves to be a strong lever”
↩︎ Prediction“Separating the agent doing the work from the agent judging it proves to be a strong lever to address this issue.”
↩︎ Checkpoint“using few-shot examples with detailed score breakdowns. This ensured the evaluator’s judgment aligned with my preferences, and reduced score drift across iterations.”
↩︎ Checkpoint