What you will be able to do
- Trace a prompt failure to a missing element: specificity, motivation, examples, structure or document placement
- Map a tool-use symptom (wrong tool, invented parameters, refused tool result) to its likely cause and fix
- Decide with an eval set whether poor results come from a model mismatch, and try effort before switching models
1.Prompt failure: when Claude does what you said, not what you meant
Many failures blamed on "the model" are actually prompt failures. Anthropic suggests treating Claude like a brilliant but new employee who lacks context on your norms and workflows. Its golden rule makes this a quick diagnostic: show the prompt to a colleague who knows little about the task and ask them to follow it. If they'd be confused, Claude will be too.
When output misses, work through the likely gaps.
- Vague request. If you want "above and beyond" behaviour, ask for it explicitly. Claude won't infer it from a vague prompt. Spell out the output format and constraints, and use numbered steps when order or completeness matters.
- Bare rule with no reason. "NEVER use ellipses" works less well than explaining that a text-to-speech engine will read the response aloud. Claude can generalise from the explanation.
- Examples that teach the wrong lesson. Few-shot examples should be relevant and diverse, so Claude doesn't pick up unintended patterns. Wrap them in <example> tags.
- Instructions mixed with data. Put instructions, context and inputs in their own XML tags to reduce misinterpretation.
- Long documents placed badly. With inputs of 20k+ tokens, put the documents near the top of the prompt, above the query, instructions and examples.
Inconsistent output is a related symptom: the same request gives differently shaped responses. Define the format precisely with JSON, XML or a template, or prefill the assistant turn so the response starts in your structure.
Checkpoint 1 of 5· Check yourself
A reporting pipeline sometimes gets clean XML from Claude and sometimes gets a friendly preamble followed by XML, which breaks the parser. Which documented fix targets this directly?
Prefilling starts Claude's turn inside your structure, which skips the preamble and enforces the format.
“Prefill the Assistant turn with your desired format. This trick bypasses Claude's friendly preamble and enforces your structure.”Source: platform.claude.com
Checkpoint 2 of 5· Exam question
A team deployed Claude Opus 4.8 behind a high-volume customer-facing chat widget that only answers short, templated questions such as order status and store hours. Users complain about slow replies, and the monthly API bill is far higher than projected. What is the most likely diagnosis?
Correct answer: A — The workload is a model mismatch: a lighter model like Claude Haiku 4.5 would meet the accuracy bar at lower cost.
- A. Correct. Simple, high-volume, templated queries are the textbook case for starting with a fast, cost-effective model; running them on the most capable and most expensive model produces exactly this symptom pattern of high latency and inflated spend for no accuracy benefit.
- B. Incorrect. Nothing in the scenario describes fabricated content — the complaints are about speed and cost, which points to model selection, not factual accuracy.
- C. Incorrect. Short templated queries are nowhere near any model's context limit, so truncation isn't a plausible cause of slow, expensive replies.
- D. Incorrect. Lengthening the system prompt for simple templated answers would increase latency and cost further without addressing the underlying model-size mismatch.
2.Tool misuse: reading the symptom back to its cause
In agentic systems, a failure usually shows up as a tool problem: the wrong tool gets called, a tool is never called, or a call has invented arguments. Anthropic's troubleshooting guide is built as symptom → likely cause → fix, and that is the order to diagnose in. Most of these causes are on your side: ambiguous descriptions, colliding names, loose schemas, how the message history is formatted, or where your instructions sit.
Diagnosing early matters more for agents than for single calls. Anthropic's write-up on its multi-agent research system notes that agents are stateful and errors compound. A small misrouted call early in a run can grow into a large change in behaviour later.
| Symptom | Likely cause | Fix |
|---|---|---|
| Claude calls tool A when you wanted tool B | Description ambiguity | Differentiate tools by WHEN to use them, not only WHAT they do |
| Claude never calls your tool | Tool name collision or overly-generic schema | Check for duplicate names; add input_examples |
| Parameter that doesn't exist in your schema | Model over-generation without strict mode | Add strict: true if the schema is in the supported subset |
| Claude calls tools sequentially when parallel would be better | Message history formatting | Send multiple tool_result blocks in ONE user message |
| Claude refuses to act on a tool result | Your instructions are inside the tool_result content | Move instructions to a user turn after the tool_result block |
Checkpoint 3 of 5· Match them up
Match each tool-use symptom to its likely cause
Tap a term, then the definition that fits it.
Each symptom points to a specific defect in tool definitions or request handling. Fix that defect before you suspect the model.
“Differentiate tools by WHEN to use them, not only WHAT they do.”Source: platform.claude.com
3.Model mismatch: proving it with evals before switching
Sometimes the prompt is sound and the tools are well defined, but results still fall short. Then the question is whether the model fits the task. Choosing a model means balancing capabilities, speed and cost, and a mismatch can go either way. A small model may lack the capability for complex reasoning. A frontier model may be more than a high-volume, simple task needs.
A model mismatch has to be shown with an eval set, not assumed. Anthropic's guidance on deciding whether to upgrade or change models starts with benchmark tests specific to your use case: having a good evaluation set is the most important step. Next, test with your actual prompts and data. Then compare models on accuracy, response quality and handling of edge cases, and weigh performance against cost.
Check two things before you change models. First, effort: on models that support the effort parameter, tuning effort is often a better lever than switching. Second, techniques can depend on the model. Where prompting guidance names a specific model, treat it as measured on that model and re-check it against your own evals before applying it elsewhere. A prompt tuned for one model can fail on another. Also look for breaking changes: if your integration forces tool use or turns thinking off, check the breaking-changes notes before moving to a model like Claude Opus 5.5.
Checkpoint 4 of 5· Put it in order
Put the documented steps for deciding whether to upgrade or change models in order
- 1.Test with your actual prompts and data
- 2.Compare performance across models for accuracy, response quality and edge cases
- 3.Create benchmark tests specific to your use case
- 4.Weigh performance and cost tradeoffs
The eval set comes first because every later comparison depends on it. Cost is weighed only after performance has been measured.
“having a good evaluation set is the most important step in the process”Source: platform.claude.com
Checkpoint 5 of 5· Exam question
An internal tool asks Claude to return a JSON object summarizing each support ticket. Across similar tickets, the field names and nesting occasionally change, breaking the downstream parser. Engineers confirm the underlying ticket data is well-formed and consistent. What is the most likely cause, and how should it be fixed?
Correct answer: A — The prompt never specifies a schema, so the fix is a fixed field list and a sample JSON response to anchor the format.
- A. Correct. When output structure varies despite consistent, well-formed inputs, the usual root cause is an underspecified prompt: without a fixed schema and example, the model reasonably varies field names and nesting from call to call, and pinning both resolves it.
- B. Incorrect. The scenario states ticket data is well-formed and consistent, ruling out conflicting source content as the cause of the format drift.
- C. Incorrect. The described symptom is structural format drift in a JSON schema, not fabricated ticket content, so quote-grounding doesn't address it.
- D. Incorrect. Rate limiting produces failed or delayed requests, not successful responses with inconsistent field structure.
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.When output quality is poor, the first fix is to upgrade to a bigger model.Why is that wrong?
Build a use-case eval set first. On models that support it, adjusting effort is often a better lever than switching, and an upgrade is justified only by a measured capability gap.
Covered in Model mismatch: proving it with evals before switching
2.Putting follow-up instructions inside the tool_result content is a good way to steer the next step.Why is that wrong?
Claude treats instructions inside tool results as potentially untrusted third-party content. Keep the tool result to data only and send instructions in a user turn after it.
Covered in Tool misuse: reading the symptom back to its cause
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practicesOfficial docs
“If they'd be confused, Claude will be too.”
↩︎ Prompt failure: when Claude does what you said, not what you meant“Think of Claude as a brilliant but new employee who lacks context on your norms and workflows.”
↩︎ Prompt failure: when Claude does what you said, not what you meant“Claude is smart enough to generalize from the explanation.”
↩︎ Prompt failure: when Claude does what you said, not what you meant“Place your long documents and inputs near the top of your prompt, above your query, instructions, and examples.”
↩︎ Prompt failure: when Claude does what you said, not what you meant“treat it as measured on that model and re-check it against your own evals before applying it to another”
↩︎ Model mismatch: proving it with evals before switching - 2.https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/increase-consistencyOfficial docs
“Precisely define your desired output format using JSON, XML, or custom templates so that Claude follows every output formatting element you require.”
↩︎ Prompt failure: when Claude does what you said, not what you meant“Prefill the Assistant turn with your desired format. This trick bypasses Claude's friendly preamble and enforces your structure.”
↩︎ Checkpoint - 3.
“Send multiple tool_result blocks in ONE user message, not one per turn.”
↩︎ Tool misuse: reading the symptom back to its cause“Claude is trained to treat instructions inside tool results as potentially untrusted third-party content.”
↩︎ Exam trap 2“Claude is trained to treat instructions inside tool results as potentially untrusted third-party content.”
↩︎ Prediction“Differentiate tools by WHEN to use them, not only WHAT they do.”
↩︎ Checkpoint - 4.
“Agents are stateful and errors compound.”
↩︎ Tool misuse: reading the symptom back to its cause - 5.
“Choosing a Claude model means balancing capabilities, speed, and cost.”
↩︎ Model mismatch: proving it with evals before switching“Compare performance across models for: Accuracy of responses Response quality Handling of edge cases”
↩︎ Model mismatch: proving it with evals before switching“If your integration forces tool use (tool_choice of type any or tool), turns thinking off”
↩︎ Model mismatch: proving it with evals before switching“Tuning effort is often a better lever than switching models.”
↩︎ Exam trap 1“having a good evaluation set is the most important step in the process”
↩︎ Checkpoint