What you will be able to do
- Check a draft ADR against a named stakeholder constraint and find the claim that leaves it unaddressed
- Rewrite determinism, latency and cost claims about Claude as measured, bounded statements with a fallback or a review trigger
- Explain why re-running a prompt, enabling streaming, or using single-agent token counts does not give the evidence a stakeholder asked for
- Add review triggers, such as model deprecation, that reopen a decision when its assumptions stop holding
Key concept
Explicit residual risk — An ADR for a Claude system says what was measured, under what conditions, what risk remains, and what happens when that risk materialises. It doesn't promise model behaviour that the model can't guarantee. A stakeholder should be able to see the trade-off they are accepting, not just the benefit the decision was chosen for.
1.Reading an ADR against the constraint it must satisfy
An architecture decision record (ADR) is reviewed by the people who will live with the decision. An auditor needs past outputs to be explainable. A support director needs replies to arrive in time. Finance needs spend to stay under a ceiling. A draft fails review when its Consequences section lists only the benefit the architect chose the design for, and is silent on the constraint a stakeholder actually stated. Fixing it starts with finding the claim that should answer that constraint, then asking whether the claim is true, measured, and bounded.
A complete ADR for a Claude system has six parts:
- context, including each stakeholder constraint written in the stakeholder's own terms
- the options considered, including the ones rejected
- the decision itself
- consequences, covering costs and risks as well as gains
- the trade-off that was accepted, and why its value justifies its cost
- review triggers that reopen the decision, such as a model deprecation
Claims about model behaviour are where drafts go wrong most often. A Claude model follows instructions by continuing a pattern, so there is always a gap between what you intended and what landed. A sentence like "the model will always…" is a commitment the system can't keep. Unsupportable SLA wording is covered in unsupportable commitments. In an ADR, the same idea is applied to a design rationale. Each behavioural claim becomes a measurement, plus the conditions it was taken under, plus the residual risk.
Anthropic's write-up of the Claude Code auto-mode classifier shows the tone to aim for. It reports a 17% false-negative rate on real overeager actions and calls that the honest number. It then says when that rate is an improvement and when it is a regression, depending on what you compare it against. That is a trade-off stated with its residual risk attached, and it makes no claim that the classifier is safe.
Checkpoint 1 of 8· Check yourself
A support director has stated a constraint: suggested replies must reach agents within 3 seconds at peak load. The draft ADR's Consequences section only says the chosen Claude model gives higher-quality suggestions. Which revision best fixes the draft?
The fix answers the stated constraint with a measurement taken under the stakeholder's conditions, the remaining risk, and what happens if the target is missed. A prompt instruction can't guarantee behaviour. Rewording the constraint, swapping in streaming, or moving the constraint to another document all leave it unanswered.
“It also means there's always a gap between what you intended and what landed”Source: academy.claude.com
2.Audit constraints: a stored record beats a re-run
"Temperature 0 makes outputs deterministic, so auditors can reproduce any recommendation" overstates what the model can do. Temperature 0 gives more conservative output that sticks to the most probable phrasing, which reduces variation. It does not promise identical text, and this applies on Anthropic's own inference service and through third-party cloud providers alike. The accurate rewording is that temperature 0 reduces output variation without guaranteeing identical text.
The audit constraint is that any past recommendation must be reviewable and explainable later. A re-run can't meet that, so the evidence has to be captured when the recommendation is made. For each recommendation, store:
- the model identifier
- the prompt version
- the inputs
- the returned output
Auditors then review that stored record. Content-moderation pipelines built on Claude use the same pattern: they keep the ruleset immutable and record the content, the extracted fields, the ruleset version, the verdict and the trace for each review, so they can answer an auditor's "why?" months later.
Pinning a dated snapshot and adding top_k set to 1 looks more rigorous, but it fails for two reasons. First, the non-determinism remains. Second, a re-run audit can outlive its model. Dated snapshots such as claude-opus-4-20250514 and claude-sonnet-4-20250514 have already been retired. Once a snapshot is retired, nobody can re-run anything against it.
Asking Claude to return its step-by-step reasoning and treating that as audit evidence fails for a related reason. The explanation is generated output, and it carries the same variability as the recommendation. Depending on configuration, returned thinking can be a summary of the raw reasoning or can be omitted and redacted. It can be part of the stored record, but it isn't proof of how the decision was reached. Having a loan officer sign off is a useful control, but it answers a different question: it records that a person approved the recommendation, not what the model recommended or on what inputs.
| Revision | What it gives the auditor | Gap |
|---|---|---|
| temperature 0 plus pinned snapshot, auditors re-run | A new output that is probably similar | Not guaranteed identical; impossible after the snapshot retires |
| Model identifier, prompt version, inputs and output stored per recommendation | The exact recommendation as it was made | None for reviewability; this is the reference record |
| Claude returns step-by-step reasoning as evidence | Generated text that may be summarized | Variable output, not a verified account of the decision |
| Loan officer sign-off as evidence | Proof that a person approved | Doesn't record what the model produced or why |
Checkpoint 2 of 8· Match them up
An auditor requires that any past lending recommendation can be reviewed and explained months later. Match each proposed ADR revision to what it actually gives the auditor.
Tap a term, then the definition that fits it.
Only a record captured when the recommendation is made shows exactly what was recommended. A re-run isn't guaranteed to match and stops working once the snapshot retires. Returned reasoning is itself variable generated output. Sign-off records a human approval, not the model's output.
“record (content, extracted fields, ruleset version, context, verdict, trace) per review; that answers an auditor's "why?" months later”Source: platform.claude.com
Checkpoint 3 of 8· Exam question
An ADR for a lending decision-support assistant built on the Claude Messages API states: "Setting temperature to 0 makes outputs deterministic, so auditors can reproduce any recommendation." The audit stakeholder's constraint is that any past recommendation must be reviewable and explainable later. Which two revisions make the trade-off and residual risk explicit? Choose two.(Select 2)
Correct answers: A, D — Store the model identifier, prompt version, inputs and returned output for each recommendation, and have auditors review that stored record rather than a re-run.; Reword the claim to say temperature 0 reduces output variation without guaranteeing identical text, and treat the stored record as the reference for audits.
- A. Correct. Keeping the model identifier, prompt version, inputs and output for each recommendation gives auditors a durable artifact to inspect later, and it works even if a new call would return different text.
- B. Pinning a snapshot and lowering sampling randomness reduces variation but does not guarantee identical output. Telling auditors they can regenerate a past recommendation keeps the overclaim and names no residual risk.
- C. Reasoning text generated by the model is not a guaranteed trace of how the output was actually produced. Documenting it as evidence of how the decision was reached overstates what it shows and states no residual risk.
- D. Correct. It replaces the determinism claim with an accurate one, that low temperature reduces variation without guaranteeing identical text. It also names the stored record, not a re-run, as the audit reference.
- E. A loan officer's sign-off shows a person reviewed the advice, but it does not preserve what the model returned or the inputs behind it. A past recommendation stays hard to review and explain, and the determinism limits go unstated.
3.Latency constraints: measure under the conditions the stakeholder named
"Opus meets the latency target" is a claim with no evidence attached. Suppose the support director's constraint is that suggested replies appear within 3 seconds at p95 during peak hours. Every word of that matters: p95 rather than average, a complete suggested reply rather than the first token, and peak load rather than a quiet afternoon. The ADR needs a number measured under those conditions.
Latency depends on the model, the prompt and the serving conditions. Latency can be influenced by the size of the model, the complexity of the prompt, and the underlying infrastructure. A single-user pilot run off-peak measures a different situation from the one the director cares about. Saying peak traffic "should stay close" to that figure is a guess presented as evidence. Extended thinking adds to the gap: higher effort levels may result in more thorough analysis but take longer.
No. Streaming lets the model start sending its response before the full output is complete. That can significantly improve perceived responsiveness, but it measures time to first token, not the time until a usable reply is on screen. Rewriting the requirement in terms of first tokens quietly changes what the director asked for. Loosening the target to 10 seconds because extended thinking is more accurate is worse, because it replaces the stakeholder's constraint with the architect's preference.
The revision that holds up has three parts:
- A p95 figure from a load test at peak concurrency.
- A plain statement of whether that figure is under 3 seconds.
- A named fallback with its cost stated.
The fallback is usually a smaller model. Claude Haiku 5.5 is the speed-critical option, so the ADR can name it and say what quality the agents give up when it is used. This is the trade-off described in model tier selection, recorded so the director can accept or reject it.
Checkpoint 4 of 8· Check yourself
The director's constraint reads: suggested replies appear within 3 seconds at p95 during peak hours. Four engineers each propose one evidence line for the ADR. Which line actually shows whether the constraint is met?
Only the peak-concurrency p95 for complete replies matches every part of the constraint. The other lines measure first tokens, a quiet single-user pilot, or an average, which are all different from what the director specified.
“The 17% false-negative rate on real overeager actions is the honest number.”Source: www.anthropic.com
Checkpoint 5 of 8· Exam question
An ADR selects Claude Opus with extended thinking for a live-chat agent-assist feature. The support director's constraint is that suggested replies must appear within 3 seconds at p95 during peak hours. The draft says only: "Opus meets the latency target." Which revision best addresses the constraint?
Correct answer: B — Add a p95 figure from a peak-concurrency load test, state whether it is under 3 seconds, and name a smaller-model fallback with its quality cost.
- A. The 3-second p95 limit was set by the support director, and an ADR author cannot loosen it to fit the chosen model. The accuracy benefit may be worth a trade-off, but that has to be proposed and agreed, not written in as a new requirement.
- B. Correct. The draft asserts a result without evidence. Measuring p95 under peak concurrency tests the exact percentile and load the director set, and it records an honest pass or fail. A named smaller-model fallback with its quality cost makes the trade-off explicit.
- C. A single-user run off-peak does not reflect contention at peak hours, so even its p95 says little about peak behavior. Expecting peak latency to stay near it is an assumption, and tail latency usually grows faster than typical latency under load.
- D. Streaming can improve perceived responsiveness, but this is still an assertion with no measurement at p95 under peak load. Extended thinking runs before the visible answer, so the first tokens may arrive later than claimed. Dropping any fallback leaves no plan if the limit is missed.
4.Cost constraints, rejected alternatives and review triggers
Multi-agent designs carry a cost that a list of benefits hides. Take an ADR that proposes an orchestrator with five parallel subagents for contract review. If its Consequences section lists only "higher accuracy through specialization", it has not addressed Finance's monthly spend ceiling per business unit. In Anthropic's own data, agents use about 4× more tokens than chat interactions, and multi-agent systems use about 15× more. Those multipliers come from one production system, not a law. They do settle the direction: splitting work across subagents raises total token use.
That rules out two tempting revisions. A note claiming five subagents use about as many tokens as a single agent is false on direction. A monthly projection that multiplies the published per-token price by single-agent token counts uses the wrong unit, so it underestimates spend while looking quantitative. What the ADR needs is a per-contract token cost measured from pilot traces of the multi-agent design itself, a statement that multi-agent runs consume more than one agent would, and a budget alert. Deferring cost to a later ADR owned by another team leaves Finance's constraint unanswered at the point of decision.
The other half of the fix is to record the trade-off as a choice. Agentic systems often trade latency and cost for better task performance, and a multi-agent design is economically viable only when the task is valuable enough to pay for the increase. So the single-agent workflow belongs in the ADR as a rejected alternative. The ADR should record the judgement that the measured accuracy gain is worth the extra spend, and set a review trigger near the spend ceiling. The structure itself is covered in orchestrator and subagent decomposition; the ADR's job is to show the price.
Review triggers are also how an ADR stays honest about the model. Anthropic deprecates and retires models to make capacity for new releases. Customers with active deployments get at least 60 days' notice, and the guidance is to test replacement models well before the retirement date. A deprecation notice for the pinned model should therefore reopen the decision.
A model upgrade should reopen it too. Anthropic moved an internal incident-response agent from Claude Opus 4 to Opus 4.5 without changing its tools, permissions or prompts. The upgraded agent then started reaching out to another agent to request a production fix, which no one had asked it to do. A latency figure, a cost estimate or a behavioural assumption measured on one model is evidence about that model only.
Checkpoint 6 of 8· Match them up
Finance has set a monthly spend ceiling per business unit. A draft ADR proposes an orchestrator with five parallel subagents for contract review. Match each proposed cost revision to its effect on Finance's constraint.
Tap a term, then the definition that fits it.
Agentic designs trade extra cost for better task performance, so only cost measured on the multi-agent design itself, with an alert near the ceiling, tells Finance what they are accepting. The other revisions get the direction wrong, use the wrong baseline, or skip the constraint.
“Agentic systems often trade latency and cost for better task performance, and you should consider when this tradeoff makes sense.”Source: www.anthropic.com
Checkpoint 7 of 8· Check yourself
An ADR's latency figure, cost estimate and behavioural assumptions were all measured on one pinned Claude model. Which review-trigger policy keeps the ADR honest?
Measurements taken on one model are evidence about that model only. Anthropic's incident-response agent began acting in new ways after a model upgrade, even though its tools, permissions and prompts did not change. So both a deprecation notice and a model upgrade should reopen the decision.
“we moved this agent from Claude Opus 4 to Claude Opus 4.5 and changed nothing else—no new tools, permissions, or prompts.”Source: claude.com
Checkpoint 8 of 8· Exam question
An ADR proposes an orchestrator with five parallel subagents on the Claude Agent SDK for contract review. Finance's stated constraint is a monthly spend ceiling per business unit. The draft's Consequences section lists only "higher accuracy through specialization." Which two revisions best address the constraint? Choose two.(Select 2)
Correct answers: B, D — Add a per-contract token cost estimate from pilot traces, note that multi-agent runs consume more tokens than one agent, and add a budget alert.; Record the single-agent workflow as a rejected alternative, judge the measured accuracy gain worth the added spend, and set a review trigger near the ceiling.
- A. Each subagent carries its own context and makes its own model calls, so five parallel subagents typically use more tokens in total than one agent. Claiming near-equal usage misstates the cost impact that a monthly spend ceiling depends on.
- B. Correct. Pilot traces give a measured per-contract token figure, the note names the higher token use of multi-agent runs, and a budget alert gives finance a control tied to its monthly ceiling.
- C. A projection built from single-agent token counts understates spend for five parallel subagents, which each consume their own tokens. A price and volume forecast on that basis cannot show whether the monthly ceiling will hold.
- D. Correct. It records the simpler single-agent alternative as weighed and rejected, states the accuracy versus spend trade-off explicitly, and sets a trigger to revisit the decision as spend nears the ceiling.
- E. Finance's ceiling is a stated constraint, so the ADR has to record the cost consequences when the architecture is chosen. Deferring them to a follow-up document leaves the decision unaddressed against that constraint.
Sources
(10)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“there's always a gap between what you intended and what landed”
↩︎ Reading an ADR against the constraint it must satisfy“It also means there's always a gap between what you intended and what landed”
↩︎ Checkpoint - 2.https://www.anthropic.com/engineering/claude-code-auto-modeSecondary source
“The 17% false-negative rate on real overeager actions is the honest number.”
↩︎ Reading an ADR against the constraint it must satisfy“Whether 17% is acceptable depends on what you're comparing against.”
↩︎ Reading an ADR against the constraint it must satisfy“The 17% false-negative rate on real overeager actions is the honest number.”
↩︎ Checkpoint - 3.
“Lower temperatures result in more conservative and deterministic outputs that stick to the most probable phrasing and answers.”
↩︎ Audit constraints: a stored record beats a re-run“Even with temperature set to 0, the results will not be fully deterministic, and identical inputs might produce different outputs across API calls.”
↩︎ Audit constraints: a stored record beats a re-run“Even with temperature set to 0, the results will not be fully deterministic, and identical inputs might produce different outputs across API calls.”
↩︎ Exam trap 1 - 4.
“record (content, extracted fields, ruleset version, context, verdict, trace) per review; that answers an auditor's "why?" months later”
↩︎ Audit constraints: a stored record beats a re-run - 5.
“claude-opus-4-20250514 | Retired | April 14, 2026 | June 15, 2026”
↩︎ Audit constraints: a stored record beats a re-run“Anthropic currently deprecates and retires models to ensure capacity for new model releases.”
↩︎ Cost constraints, rejected alternatives and review triggers“Test your applications with newer models well before the retirement date of your current model.”
↩︎ Cost constraints, rejected alternatives and review triggers - 6.https://platform.claude.com/docs/en/api/messagesOfficial docs
“Reflects the raw reasoning the model produced, not the (possibly shorter) summarized thinking text returned in the response body.”
↩︎ Audit constraints: a stored record beats a re-run“When set to omitted, thinking content is redacted but a signature is returned for multi-turn continuity.”
↩︎ Audit constraints: a stored record beats a re-run“Higher effort levels may result in more thorough analysis but take longer.”
↩︎ Latency constraints: measure under the conditions the stakeholder named“Reflects the raw reasoning the model produced, not the (possibly shorter) summarized thinking text returned in the response body.”
↩︎ Exam trap 2 - 7.https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-latencyOfficial docs
“Latency can be influenced by various factors, such as the size of the model, the complexity of the prompt, and the underlying infrastructure”
↩︎ Latency constraints: measure under the conditions the stakeholder named“This can significantly improve the perceived responsiveness of your application”
↩︎ Latency constraints: measure under the conditions the stakeholder named“For speed-critical applications, Claude Haiku 5.5 offers the fastest response times while maintaining high intelligence.”
↩︎ Latency constraints: measure under the conditions the stakeholder named“Streaming is a feature that allows the model to start sending back its response before the full output is complete.”
↩︎ Exam trap 3 - 8.
“multi-agent systems use about 15× more tokens than chats”
↩︎ Cost constraints, rejected alternatives and review triggers“For economic viability, multi-agent systems require tasks where the value of the task is high enough to pay for the increased performance.”
↩︎ Cost constraints, rejected alternatives and review triggers“multi-agent systems use about 15× more tokens than chats”
↩︎ Exam trap 4 - 9.https://www.anthropic.com/engineering/building-effective-agentsSecondary source
“Agentic systems often trade latency and cost for better task performance, and you should consider when this tradeoff makes sense.”
↩︎ Cost constraints, rejected alternatives and review triggers - 10.https://claude.com/blog/ciso-guide-to-agentic-aiSecondary source
“with each model release, the agent got smarter.”
↩︎ Cost constraints, rejected alternatives and review triggers“Instead, our jobs are to make agentic risk legible and bounded.”
↩︎ Key concept“we moved this agent from Claude Opus 4 to Claude Opus 4.5 and changed nothing else—no new tools, permissions, or prompts.”
↩︎ Checkpoint