CertSafari
    Anthropic Claude Certified Architect – Professional (CCAR-P) - December 7 onwards· Lessons

    Domain 4 · Lesson 21/39

    Evaluation coverage gaps: finding the metric that would have caught it

    Evaluation Coverage Gaps

    17 min read
    2.43% of exam
    8 sources
    Published 18 Nov 2026
    Docs as of 18 Nov 2026

    What you will be able to do

    • Explain why a suite that passed can still let a production failure through, in terms of a missing metric dimension or a wrong grading target
    • Name the missing dimension (accuracy, groundedness, latency, cost, safety, security) behind a described production incident
    • Tell when the final reply is the wrong thing to grade and the transcript or outcome is the right one
    • Specify a replacement metric with a case set, a grader, a threshold and a release gate

    Key concept

    Evaluation coverage gap — A failure dimension or grading target that the pre-release suite never measured. This lets a release pass every test and still fail in production. You close the gap by adding the metric that is sensitive to that failure, not by adding more cases to the metric that already passed.

    1.Why a passing suite still ships failures

    An evaluation suite only detects the failures it measures. A suite can be large, automated and green on every run, and still be blind to the failure users hit first. The reason is usually not that it had too few cases. It never looked along the dimension where the failure lives. A post-incident review asks one question: which metric, run before release, would have turned red?

    Most production systems need several success criteria at once. A single accuracy number can't stand in for all of them. A well-formed criterion for a sentiment classifier sets a quality floor, a safety floor and a latency target side by side, all measured on the same held-out set: an F1 score of at least 0.85, 99.5% non-toxic outputs, and 95% of responses under 200ms. Remove any one line and that dimension can regress silently while the others stay green.

    Gaps come in two kinds. A dimension gap means a whole axis was never measured, such as latency, groundedness or injection resistance. A grading-target gap means the right axis was measured on the wrong artifact. The classic case is grading an agent's final reply when the failure is in what it did along the way. Either kind produces a release that passes everything it was tested on.

    Metric dimensions, the production failure each one catches, and a metric that would have caught it
    DimensionTypical production failureMetric that detects it pre-release
    AccuracyWrong label or answer on in-scope inputsExact match or F1 against golden answers on a held-out set
    GroundednessConfident statement found nowhere in the provided contextClaim-level faithfulness against the retrieved passages, including out-of-corpus questions
    LatencyUsers wait; first-response target missedTime to first token and response-time percentiles against the product target
    CostSpend per task rises after a configuration changeToken usage and cost per task on a static bank of tasks
    SafetyToxic or harmful output reaches usersRate of flagged outputs over many trials, with a maximum allowed rate
    SecurityAgent follows injected instructions or takes unauthorized actionsInjection-resistance rate on a red-team set; trace check for unauthorized tool calls

    Mature evaluation programs have learned this the hard way. In one incident review, Anthropic described running benchmarks, safety evaluations and canary deployments, and still finding that the evaluations didn't capture the degradation users reported. Their fix was more sensitive evaluations, ones that reliably tell a working implementation from a broken one. More of the same evaluations would not have helped.

    Checkpoint 1 of 8· Check yourself

    A sentiment classifier's release suite measured only F1 on 10,000 held-out posts. It scored 0.91 and the release shipped on a larger model. Within a day, users complained that replies were too slow. Which pre-release addition would have turned the suite red?

    Sources12

    2.Accuracy against reference answers cannot see fabrication

    Accuracy and groundedness answer different questions. Accuracy asks whether the output matches a known correct answer. Groundedness asks whether every statement in the output is supported by the context the model was given. A model can be accurate on every question the knowledge base answers and still invent a policy when asked about something the knowledge base never mentions. A suite made only of answerable questions never presents that situation.

    An exact-match grader sees only whether the output equals the golden answer. It has no view of the context the answer should rest on.python
    def evaluate_exact_match(model_output, correct_answer):
        return model_output.strip().lower() == correct_answer.lower()

    The missing metric has two parts, and both are needed. The case set has to include questions the corpus cannot answer. Irrelevant or nonexistent input data is a standard edge case, and out-of-corpus questions are exactly where a retrieval system is tempted to fill the gap. The grader has to break the answer into claims and check each one against the retrieved passages. That measures how well the model uses provided context, rather than how closely it matches a reference answer. A run that confidently asserts something no passage supports fails, even if the claim sounds plausible.

    The tempting alternatives each measure a neighbouring property. Retrieval hit rate at k checks whether the right document came back. It grades the retriever, not the answer, and on an out-of-corpus question there is no gold document to find. Answer relevancy scores whether the reply addresses the user's request. A fabricated 60-day refund window is perfectly on-topic, so relevancy passes it. Exact match on more in-corpus questions only adds weight to a dimension that was already green.

    Checkpoint 2 of 8· Check yourself

    After the fabricated refund window, a team adds a grader that splits each answer into claims and checks every claim against the retrieved passages. They run it only on their existing questions, which the knowledge base answers. It passes at 99%. What change would make this suite catch the next fabrication before release?

    Checkpoint 3 of 8· Exam question

    A support assistant built on the Claude Messages API with retrieval told a customer that refunds are allowed within 60 days, a policy that appears nowhere in the help center. The pre-release suite scored answers against reference answers on questions the knowledge base covers, and all of them passed. As the architect running the post-incident review, which missing metric would have detected this failure before release?

    Sources31

    3.Latency and cost: dimensions an accuracy gain can hide

    A configuration change that raises quality often moves latency and cost in the wrong direction at the same time. Model size is one of the main factors in response time. Extended thinking adds reasoning tokens before the answer, which is why thinking budgets are tuned against both latency and quality needs. A team that moves to a larger model with thinking enabled, and re-runs only its accuracy suite, will see the improvement and none of the cost of it.

    For a streaming chat product, the metric that matches user experience is time to first token (TTFT): the time from sending the prompt to receiving the first token of the response. If the product promises a first response within 2 seconds, the release gate is TTFT measured against that target, together with end-to-end response time. Both should be expressed as percentiles, the way a criterion like "95% response time < 200ms" is written. Latency also depends on the infrastructure serving the model, so the measurement has to run under conditions that mirror real-world traffic. A quiet offline run is not enough.

    Two plausible substitutes fail this test. A mean wall-clock time from one sequential run hides the slow tail that users complain about, and it never sees production load. Mean output tokens per response tracks verbosity. It is a proxy that moves with latency sometimes, but it is not the target the product committed to. Neither one turns red when the 2-second promise is broken. Re-running accuracy on a bigger sample is a different dimension altogether.

    Cost gaps behave the same way. Once a static bank of tasks exists, latency, token usage, cost per task and error rates can all be tracked on it at no extra design effort. A release that doubles tokens per task then shows up as a number before it shows up on an invoice. Budget is a success criterion in its own right, alongside per-call cost and frequency of usage. How to reduce latency or cost once you have measured it belongs to cost and latency optimization.

    Checkpoint 4 of 8· Match them up

    Your team moves a streaming chat product to a larger model with extended thinking. The product promises a first response within 2 seconds. Match each candidate release metric to what it would actually tell you about this change.

    Tap a term, then the definition that fits it.

    Checkpoint 5 of 8· Exam question

    A chat product moved from a smaller Claude model to a larger one with extended thinking enabled, using the Messages API with streaming. The pre-release suite showed higher answer accuracy and the team shipped. Within a day, users complained about long waits and the 2-second first-response target was missed. Which metric would have detected this before release?

    Sources4516

    4.Safety, security and grading the trace instead of the reply

    For agents, a grader has three possible targets: the final reply, the transcript (every tool call, intermediate result and reasoning step), and the outcome (the final state of the environment). An agent that takes actions can do harm that never shows up in what it says. Code-based graders can verify tool calls directly, checking which tools were used and with what parameters. That makes "fail any run whose trace contains a delete_record call the user did not authorize" a cheap, objective, reproducible check.

    Security is its own dimension because its adversary is not the ordinary user. Indirect prompt injection is the case where the user is trusted but the model processes third-party content, such as webpages, emails, documents or tool results, that carries adversarial instructions. Benign task-success cases never contain such content, however many of them there are. The suite has to add adversarial cases: retrieved pages that embed instructions to call destructive tools, plus a red-team set covering both direct and indirect injection. The defenses themselves are covered in prompt injection controls. Here the concern is measuring whether they hold.

    Safety and security metrics become release gates when they are written as rates with thresholds. A safety criterion can be as concrete as fewer than 0.1% of outputs out of 10,000 trials flagged for toxicity. An injection-resistance rate over the red-team set, with a minimum pass rate, works the same way. Pair the rate, which tracks overall resistance, with a trace check that fails any run containing an unauthorized destructive call. One measures how often the agent resists; the other catches the specific harm.

    Checkpoint 6 of 8· Match them up

    After an injected instruction on a retrieved web page led a support agent to call delete_record, the team reviews five proposed additions to the suite. Match each addition to what it would actually tell them before the next release.

    Tap a term, then the definition that fits it.

    Checkpoint 7 of 8· Exam question

    A coding-ops agent built on the Agent SDK browses web pages and can call a delete_record tool. In production, a retrieved page contained hidden text telling the agent to delete customer records, and the agent complied. The pre-release suite only measured task success on benign requests. Choose two additions that would have detected this failure before release.(Select 2)

    0 of 2 selected

    Sources716

    5.Specifying the metric that closes the gap

    Naming the missing dimension is half the review. The other half is writing the metric precisely enough that the next release cannot pass without it. Vague criteria such as "the bot should not make things up" or "it should be fast" cannot gate anything. A usable specification is specific and measurable in the same sense as any success criterion.

    Build the specification in this order:

    1. Name the dimension the incident belongs to, and check whether the gap was the dimension itself or the grading target.
    2. Define the case set that exposes it: out-of-corpus questions, injected pages, production-like traffic. Source these cases from the failure you actually saw.
    3. Choose the grader and its target. Use a code check on tool calls, a claim-level check against context, or a timing measurement, applied to the reply, the trace or the outcome.
    4. Set a threshold, such as a minimum rate, a maximum rate or a percentile target, based on the product commitment.
    5. Make it a gate: the full suite must pass before a new version is promoted.

    Run-to-run variation matters for gates. Model outputs vary between runs, so a rate measured over several trials per task is more trustworthy than a single pass. How to compare two configurations rigorously belongs to experimental design. Rerunning the expanded suite periodically also catches drift as workflows and models change. Every incident should leave behind at least one permanent case that would now fail.

    Checkpoint 8 of 8· Check yourself

    Four drafts of a release requirement follow the incident where an injected page led the agent to call delete_record. Which draft is specified well enough to block the next version?

    Sources186

    Sources

    (8)

    Every claim above is drawn from one of these pages, quoted as it was written on the date shown.

    1. 1.
      “Most use cases need multidimensional evaluation along several success criteria.”
      ↩︎ Why a passing suite still ships failures
      “How effectively does the model use provided context?”
      ↩︎ Accuracy against reference answers cannot see fabrication
      “What is your budget for running the model?”
      ↩︎ Latency and cost: dimensions an accuracy gain can hide
      “Less than 0.1% of outputs out of 10,000 trials flagged for toxicity by the content filter.”
      ↩︎ Safety, security and grading the trace instead of the reply
      “Specific: Clearly define what you want to achieve.”
      ↩︎ Specifying the metric that closes the gap
      “Most use cases need multidimensional evaluation along several success criteria.”
      ↩︎ Key concept
      “How effectively does the model use provided context?”
      ↩︎ Exam trap 3
      “Example edge cases Irrelevant or nonexistent input data”
      ↩︎ Checkpoint
    2. 2.
      “These issues exposed critical gaps that we should have identified earlier.”
      ↩︎ Why a passing suite still ships failures
      “we’ve developed evaluations that can more reliably differentiate between working and broken implementations”
      ↩︎ Why a passing suite still ships failures
    3. 3.
      “can sometimes generate text that is factually incorrect or inconsistent with the given context”
      ↩︎ Accuracy against reference answers cannot see fabrication
    4. 4.
      “This metric measures the time it takes for the model to generate the first token of the response, from when the prompt was sent.”
      ↩︎ Latency and cost: dimensions an accuracy gain can hide
      “Latency can be influenced by various factors, such as the size of the model, the complexity of the prompt”
      ↩︎ Latency and cost: dimensions an accuracy gain can hide
    5. 5.
      “start with a larger budget of 16,000 tokens or more and adjust to your latency and quality needs”
      ↩︎ Latency and cost: dimensions an accuracy gain can hide
      “start with a larger budget of 16,000 tokens or more and adjust to your latency and quality needs”
      ↩︎ Exam trap 2
    6. 6.
      “latency, token usage, cost per task, and error rates can be tracked on a static bank of tasks”
      ↩︎ Latency and cost: dimensions an accuracy gain can hide
      “Tool calls verification (tools used, parameters)”
      ↩︎ Safety, security and grading the trace instead of the reply
      “The outcome is the final state in the environment at the end of the trial.”
      ↩︎ Safety, security and grading the trace instead of the reply
      “Because model outputs vary between runs, we run multiple trials to produce more consistent results.”
      ↩︎ Specifying the metric that closes the gap
      “Source realistic tasks from the failures you see.”
      ↩︎ Specifying the metric that closes the gap
      “The outcome is the final state in the environment at the end of the trial.”
      ↩︎ Exam trap 4
      “A transcript (also called a trace or trajectory) is the complete record of a trial, including outputs, tool calls, reasoning, intermediate results”
      ↩︎ Prediction
      “Each grader evaluates some portion of either the transcript or the outcome.”
      ↩︎ Checkpoint
    7. 7.
      “Indirect prompt injection, where the user is trusted but Claude processes third-party content (webpages, emails, documents, tool results) that contains adversarial instructions.”
      ↩︎ Safety, security and grading the trace instead of the reply
      “Indirect prompt injection, where the user is trusted but Claude processes third-party content (webpages, emails, documents, tool results) that contains adversarial instructions.”
      ↩︎ Exam trap 1
    8. 8.
      “Require the full evaluation suite to pass before promoting new versions.”
      ↩︎ Specifying the metric that closes the gap

    Ready to test yourself?

    Practise Anthropic Claude Certified Architect – Professional (CCAR-P) - December 7 onwards in quiz mode.

    Spotted a mistake, or was something unclear? Tell us.