What you will be able to do
- Use production cost, latency, failure and eval data to decide whether a running agent should move part of its traffic onto fixed workflows
- Decide when a fixed workflow should gain a bounded agent step, and how to measure whether the change worked
- Judge whether a multi-agent system has earned its token overhead, and decide what the default path should be
- Choose among the Agent SDK, the Client SDK, Managed Agents and third-party frameworks, and set the standards that a team's reusable patterns must meet
Key concept
Workflow–agent spectrum — A workflow runs the model through code paths that you fix in advance. An agent lets the model choose its own steps and tool calls. At scale you choose a position on that spectrum for each slice of traffic, based on measured cost, latency and quality, and you add model-driven control only where fixed paths fail.
1.Workflows, agents, and the tradeoff between them
Agentic systems come in two kinds, and the difference between them is who controls the path. In a workflow, your code decides the sequence: call the model, check the output, call it again, branch on a classification. In an agent, the model decides which tool to call next and when it is finished. Both are agentic systems. They differ in cost, in latency, and in how predictable their behaviour is.
Every step toward agency costs something. Agentic systems often give up latency and cost in exchange for better task performance. The standing advice is to find the simplest solution that works and add complexity only when you need it. Sometimes that means no agentic system at all, because a single well-built LLM call with retrieval and in-context examples is often enough. When you do need more, workflows give predictability and consistency on well-defined tasks. Agents are the better choice when the system needs flexibility and model-driven decisions.
| Pattern | What it does | Use it when |
|---|---|---|
| Prompt chaining | Each call processes the previous call's output, with optional programmatic gates between steps | The task splits cleanly into fixed subtasks, and you accept extra latency for higher accuracy |
| Routing | Classifies the input and sends it to a specialised follow-up prompt, tool set or model | The input falls into distinct categories, and the classification can be done accurately |
| Parallelization | Sectioning runs independent subtasks at the same time. Voting runs the same task several times | Subtasks can run in parallel for speed, or you need several attempts for higher confidence |
| Orchestrator-workers | A central LLM breaks the task down at runtime and delegates the pieces | You cannot predict which subtasks the input will need |
| Evaluator-optimizer | One call generates and another critiques, in a loop | You have clear evaluation criteria and refinement produces a measurable improvement |
The "use it when" column turns architecture choice into something you can measure. Each condition can be checked against data: whether the subtasks are actually fixed, whether a classifier separates the categories accurately, whether a refinement loop raises the score. For a system that is already running, those questions are answered by its success criteria and the evaluations built against them, not by what the team prefers.
Checkpoint 1 of 9· Match them up
Your team measures each slice of traffic before picking a design. Match each workflow pattern to the condition that justifies using it.
Tap a term, then the definition that fits it.
Each pattern has a condition you can check against data. If the subtasks are fixed, use chaining. If the categories are separable, use routing. If the subtasks are unpredictable, use orchestrator-workers. If refinement measurably helps, use evaluator-optimizer. Orchestrator-workers is the only one of these where the model decides the subtasks at runtime.
“This workflow is well-suited for complex tasks where you can’t predict the subtasks needed”Source: www.anthropic.com
2.When production data says an agent should become a workflow
Agents that run in production often turn out to be doing predictable work. Traces show most requests following a small number of repeated tool sequences, and the model rediscovers the same path every time at full loop cost. That repetition is evidence that the condition for a workflow holds: the task, at least for that slice of traffic, splits into fixed subtasks.
The deciding test is a comparison against a baseline. Build the fixed pipeline for the common paths, run it offline on the same tasks, and compare its quality with the agent's. If the pipeline matches the agent on those paths, the agent's flexibility is buying nothing there. Moving those paths to prompt chains then cuts cost and latency and makes behaviour more consistent, with no loss in quality.
Moving the common paths does not mean removing the agent. The right structure is usually a router in front. A classifier sends requests that match a known path to its scripted chain and sends everything else to the existing agent loop. This works because routing separates concerns: tuning one prompt for one kind of input can hurt performance on other inputs. The open-ended tail still gets the flexibility it needs.
Three tempting alternatives keep the cost structure you are trying to escape:
- A larger model in the same open loop still pays for model-driven planning on every request. It may take fewer steps, but the path is still not fixed.
- Step-by-step playbooks written into the system prompt still run the free-form loop. The model decides whether to follow them, so you get neither the cost savings nor the consistency of code-defined paths.
- A hard cap on tool calls cuts off the loop without removing the unnecessary planning. Requests that legitimately need more steps return worse answers.
Checkpoint 2 of 9· Check yourself
Traces from a production support agent show that about 70% of requests follow one of three repeated tool sequences. An offline eval finds that a fixed pipeline for those three sequences matches the agent's quality on them. The remaining 30% vary widely. What change should the team make?
On the predictable slice, the agent's flexibility adds nothing, so a router moves that slice to fixed chains for lower cost and more consistent behaviour, and the open-ended tail keeps the agent. Playbooks and tool-call caps leave the free-form loop and its cost in place. Replacing the agent entirely takes flexibility away from the 30% of requests that need it.
“workflows offer predictability and consistency for well-defined tasks, whereas agents are the better option when flexibility and model-driven decision-making are needed at scale”Source: www.anthropic.com
Checkpoint 3 of 9· Exam question
Heron Grocers runs a support-triage agent in production for its online grocery delivery service. The agent works in a free-form tool loop, choosing each next tool call itself. Traces show 92% of tickets, such as missing items and late-delivery credits, follow one of six repeatable resolution paths, averaging 11 tool calls, $0.31 per ticket and 40 s p95 latency. On those paths the agent calls the same tools in nearly the same order every time, and every call is needed to resolve the ticket. Offline, a fixed pipeline scores the same as the agent on those paths. A short trial with a larger model produced similar call counts on these tickets. The remaining 8% are unusual cases that sometimes need many exploratory calls. Leadership wants lower cost and latency without losing quality on any ticket. What is the best architectural change?
Correct answer: A — Add a classifier that sends tickets matching the six known paths to scripted prompt chains, routing the remaining tickets to the existing loop.
- A. Roughly nine in ten tickets follow known paths where a scripted flow matches the agent's quality, so moving them to fixed chains removes the loop's extra calls, spend and delay. Tickets outside those paths still get the flexible loop.
- B. A larger model costs more per token and still chooses each step itself, so the loop's overhead on predictable tickets remains. The offline result shows a scripted flow already matches the agent there, so the extra capability buys nothing on the common paths.
- C. Playbooks in the prompt are only guidance. The loop still decides each step, can deviate, and keeps paying for open-ended reasoning on tickets whose route is already known. This does not change the architecture or guarantee the shortcut is taken.
- D. Typical tickets already take about eleven tool calls, so a limit of five cuts off the common paths before they finish. That lowers quality on exactly the tickets where a fixed flow would have matched the agent, and it does not separate predictable tickets from open-ended ones.
3.When a workflow should gain a bounded agent step
Pressure can also run the other way. A fixed workflow that keeps failing on inputs it never anticipated is showing that one of its stages has subtasks you cannot predict. The clearest symptom is a fix that adds another conditional branch every time. Each branch encodes one more order of lookups someone has now seen, and the next new input breaks it again.
This is the condition the orchestrator-workers pattern was built for. Its defining feature is flexibility: the model chooses the subtasks for each input rather than following a predefined list. Extending a routing table does not escape the problem, because routing assumes distinct categories that are known in advance, and these failures come from orders that nobody anticipated.
The defensible move is narrow. Put one agent step at the failing stage and leave the stages that work as code. Give that step a small tool allowlist containing only the lookups it needs, plus a cap on turns that you set on the loop. Handing it the full toolset is a mistake: an agent with too many tools, often 20 or more, has trouble picking the right one. Tool access is a permissions decision, so set which tools run automatically and which need approval rather than inheriting everything.
Measure the change on slices, not just in aggregate. Compare failure rates before and after on the inputs that were failing and, separately, on the inputs that were already succeeding. An overall failure rate can improve while the slice that used to pass quietly gets worse. That is why replacing the whole pipeline with one agent and comparing a single number against the old rate is weak evidence. Edge-case handling is a measurable criterion in its own right, so track it as one.
Checkpoint 4 of 9· Check yourself
An invoice-processing workflow has 9 code-defined stages. Stage 4 keeps failing on invoices that need lookups in an order nobody planned for, and each fix so far has added one more conditional branch. The stage has access to 30 tools. Which change is best supported?
The failures show that stage 4's subtasks cannot be predicted, so model-driven control belongs only at that stage, with a small set of tools, because agents with too many tools choose poorly. A routing step still assumes the categories are known in advance. Replacing the whole pipeline and checking one overall number can hide regressions on inputs that used to pass. A larger model leaves the fixed branches in place.
“An agent with too many tools (often 20+) struggles to select the appropriate one.”Source: claude.com
Checkpoint 5 of 9· Exam question
Lantern Analytics runs a 14-step document-extraction workflow in production for client onboarding packets, and 18% of runs fail. Failure analysis shows most failures come from inputs that need extra lookups in an order the pipeline never anticipated, for example a field that points to a second record, which points to a third. These failures cluster at a single stage. The other stages complete cleanly on the remaining documents, at stable and low cost per run. Over two quarters the team has fixed each failure pattern by adding another conditional branch. Each branch covers only the pattern it was written for, and new patterns keep appearing. The operations lead wants these documents to stay automated, wants cost and risk kept bounded, and wants evidence that any change helps without hurting the stages that work today. What is the most defensible next step?
Correct answer: C — Add one agent step with a small tool allowlist and a turn cap at the failing stage, then compare failure rates before and after on both slices.
- A. Failures cluster in one kind of input, so the other stages of the pipeline work. Replacing all 14 steps with one agent puts the healthy majority of documents under loop cost and run-to-run variance. A single overall failure number would also hide where the new design is worse.
- B. A router with fixed lookup sequences is still pre-written branching. The failing inputs need lookups in an order nobody anticipated, so each new sequence covers only the cases already seen. The table grows, complexity rises, and the failure rate stays high.
- C. Autonomy is used only where the failure analysis shows fixed logic cannot cope, and the allowlist and turn cap keep cost and risk limited. Measuring both the affected inputs and the rest before and after shows whether the change helps without harming the stages that work.
- D. Human review is a sound fallback, but making it the permanent answer for roughly a fifth of volume gives up on automation before testing whether a constrained agent could resolve those inputs. The failure data supports trying a targeted fix first, not committing to ongoing reviewer cost.
4.Deciding whether multi-agent has earned its cost
Multi-agent architectures carry a large fixed overhead. In Anthropic's testing they typically use 3–10x more tokens than a single agent for the same task. The tokens go to context duplicated across agents, coordination messages between them, and summaries passed at each handoff. Each extra agent is also another point of failure and another set of prompts to maintain. The mechanics of orchestrators and subagents are covered in multi-agent orchestration. The question here is whether to keep them.
Multi-agent designs consistently beat a single agent in three situations:
- Context pollution, where one subtask fills the context with material that degrades reasoning on the next one.
- Work that genuinely runs in parallel, such as research across many independent sources.
- Specialisation, where focused tool sets and prompts fix tool-selection errors.
Outside these three situations, coordination costs usually outweigh the benefits. Anthropic has seen teams spend months on elaborate multi-agent systems, then find that better prompting on a single agent gave the same results.
Suppose an eval shows a single agent with the same tools scoring about as well as the multi-agent system on most production queries, with the gap showing up only on broad queries that can be split up. That calls for a split default. Make the single agent the default path, and have a classifier send only fan-out-friendly queries to the orchestrator. Retiring the orchestrator throws away the quality you measured on exactly the queries where it wins. Keeping multi-agent as the default and running subagents on a cheaper model lowers the price per token but leaves the duplicated context and coordination messages that cause the overhead.
Check the cheaper fixes before adding agents at all. An agent that is struggling with 15–20 or more tools may do better with the Tool Search Tool, which loads tool definitions on demand, than with a split into specialists. Context pressure is increasingly handled by compaction, which is covered in context and memory architecture. Some work should never be split: sequential phases of the same feature, tightly coupled components, and anything that depends on shared state.
Checkpoint 6 of 9· Match them up
A platform team gets four requests to split a single agent into several agents. For each symptom, pick the response that the multi-agent tradeoffs support.
Tap a term, then the definition that fits it.
Multi-agent designs pay off only for context pollution, truly parallel work or specialisation. Even then, try the cheaper fixes first: Tool Search for tool overload and compaction for context pressure. Sequential, tightly coupled work should never be split. Independent research is the parallel case, and what it gains is thoroughness, not speed.
“Before adopting a multi-agent architecture, consider using the Tool Search Tool, which lets Claude dynamically discover tools on-demand rather than loading all definitions upfront.”Source: claude.com
Checkpoint 7 of 9· Exam question
Kestrel Housing runs a multi-agent research system that helps its leasing analysts compare neighborhoods, rent trends and tenant-demand data. An orchestrator fans each query out to parallel subagents, and a run uses about 15x the tokens of a single-agent run. Provider capacity is not the problem: the account's rate limits have never throttled a request, but finance flags the token spend per answer as unsustainable. The team ran an eval of 400 production queries against a single agent with the same tools. The single agent scored within one point on 90% of them. The gap appears only on broad, parallelisable queries, where the multi-agent version clearly wins. Analysts say those broad queries drive their most valuable reports. What should the team do?
Correct answer: B — Make a single agent the default for production traffic and send only fan-out-friendly queries, flagged by a classifier, to the orchestrator.
- A. Higher rate limits solve a capacity problem, not a value problem. Most queries would still pay the roughly 15x token premium although a single agent scores within a point on 90% of them, so cost per unit of quality stays poor.
- B. Routing by query shape applies the expensive pattern only where the eval shows a gain. Broad, parallelisable queries keep the multi-agent quality advantage, while the 90% where a single agent matches it avoid the 15x token cost.
- C. Retiring the orchestrator gives up the measured quality gain on broad, parallelisable queries. The eval supports splitting traffic by query type, not removing multi-agent for everyone and accepting a known loss.
- D. Smaller subagent models cut the price per token but not the fan-out and coordination volume, so simple queries still run the full multi-agent pipeline. It also risks the quality that justified fan-out on broad queries, and it does not separate the two query populations.
5.Selecting the harness and framework
Every agent runs inside a harness: the system that processes inputs, orchestrates tool calls and returns results. When you evaluate an agent, you are evaluating the model and the harness together, so choosing a harness is an architecture decision. Anthropic's options differ mainly in who runs the loop and where it runs.
| Option | Who runs the agent loop | Choose it when |
|---|---|---|
| Client SDK | You write the tool loop, or use the client SDK's beta tool runner | You want direct API access and full control of every call, as with most workflows |
| Agent SDK | A library that runs Claude Code in a process you operate | You want built-in tools, permissions, sessions, hooks and subagents inside your own Python or TypeScript application |
| Managed Agents | Anthropic hosts the harness, with sessions in an Anthropic-managed or self-hosted sandbox | You want the loop hosted and configured through the Claude API |
| Claude Code CLI | The terminal interface | Interactive development or one-off tasks, not embedding in a service |
Third-party frameworks such as Strands Agents, Rivet and Vellum make it easy to start because they handle model calls, tool parsing and chaining. The same abstraction layers can hide the underlying prompts and responses, which makes failures harder to debug. They also make it tempting to add complexity a simpler setup would not need. Before a platform team approves a framework, two conditions should hold. The team must understand the code it runs, because wrong assumptions about what happens under the hood are a common source of customer error. And the prompts and responses must stay visible for debugging. Many patterns take only a few lines of direct API code, so starting with the API directly is a reasonable default.
Checkpoint 8 of 9· Match them up
Four teams describe what they need from an agent harness. Pair each need with the Anthropic option that fits it.
Tap a term, then the definition that fits it.
The options differ mainly in who runs the loop and where it runs. With the Client SDK you write the loop yourself. The Agent SDK runs Claude Code's loop as a library inside your own process. Managed Agents is a loop that Anthropic hosts. The CLI is for interactive work in a terminal, not for embedding in a service.
“Have Anthropic host the agent, configured through the Claude API | Managed Agents”Source: code.claude.com
6.Standards for a reusable pattern library
A shared pattern library spreads whatever architecture habits it contains, good or bad. If it only offers templates, teams will reach for orchestrator-workers when a prompt chain would do. Each pattern should therefore ship with the information needed to decide whether to use it, not just the code to run it.
Each pattern in the library should include:
- A when-to-use condition that can be checked against data, such as fixed subtasks for chaining, accurately classifiable categories for routing, or clear criteria plus measurable improvement for evaluator-optimizer.
- Success criteria that are specific and measurable, such as an F1 threshold on a held-out set rather than "performs well".
- A baseline eval on a fixed bank of tasks that tracks latency, token usage, cost per task and error rate, so every adopter starts with a regression test.
- The packaging that lets teams load the pattern consistently. In the Agent SDK, plugins bundle skills, agents, hooks and MCP servers, and load them by local path.
Baselines make the earlier decisions repeatable. The same task bank that justified a pattern can show, after a model upgrade or a cost audit, whether the pattern still earns its complexity. These thresholds move as models improve: today's limits on tool counts and context are practical guidelines, not fixed constraints. A team with evals can re-run the comparison and move a slice of traffic up or down the workflow–agent spectrum in days. A team without evals is guessing. How to design those suites is covered in evaluation system design.
Checkpoint 9 of 9· Check yourself
After a model upgrade, a cost audit flags the orchestrator-workers entry in a shared pattern library. That entry was justified a year ago against a fixed bank of tasks. How should the platform team decide whether the pattern still earns its complexity?
Each pattern ships with a baseline eval on a fixed task bank. Re-running it after an upgrade or audit shows whether the pattern still beats a simpler design, and these thresholds change as models improve. A survey does not produce measurable criteria. A cheaper model lowers the price per token but does not test whether the architecture is still justified.
“latency, token usage, cost per task, and error rates can be tracked on a static bank of tasks.”Source: www.anthropic.com
Sources
(5)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.https://www.anthropic.com/engineering/building-effective-agentsSecondary source
“Agentic systems often trade latency and cost for better task performance, and you should consider when this tradeoff makes sense.”
↩︎ Workflows, agents, and the tradeoff between them“systems where LLMs dynamically direct their own processes and tool usage”
↩︎ Workflows, agents, and the tradeoff between them“finding the simplest solution possible, and only increasing complexity when needed”
↩︎ Workflows, agents, and the tradeoff between them“This workflow is ideal for situations where the task can be easily and cleanly decomposed into fixed subtasks.”
↩︎ When production data says an agent should become a workflow“Without this workflow, optimizing for one kind of input can hurt performance on other inputs.”
↩︎ When production data says an agent should become a workflow“subtasks aren't pre-defined, but determined by the orchestrator based on the specific input.”
↩︎ When a workflow should gain a bounded agent step“they often create extra layers of abstraction that can obscure the underlying prompts and responses, making them harder to debug.”
↩︎ Selecting the harness and framework“If you do use a framework, ensure you understand the underlying code.”
↩︎ Selecting the harness and framework“This workflow is particularly effective when we have clear evaluation criteria, and when iterative refinement provides measurable value.”
↩︎ Standards for a reusable pattern library“Workflows are systems where LLMs and tools are orchestrated through predefined code paths.”
↩︎ Key concept“workflows offer predictability and consistency for well-defined tasks, whereas agents are the better option when flexibility and model-driven decision-making are needed at scale”
↩︎ Exam trap 1“Incorrect assumptions about what's under the hood are a common source of customer error.”
↩︎ Exam trap 5“This workflow is well-suited for complex tasks where you can’t predict the subtasks needed”
↩︎ Checkpoint“workflows offer predictability and consistency for well-defined tasks, whereas agents are the better option when flexibility and model-driven decision-making are needed at scale”
↩︎ Checkpoint - 2.
“Building a successful LLM-based application starts with clearly defining your success criteria and then designing evaluations to measure performance against them.”
↩︎ Workflows, agents, and the tradeoff between them“A/B testing: Compare performance against a baseline model or earlier version.”
↩︎ When production data says an agent should become a workflow“Edge case analysis: Percentage of edge cases handled without errors.”
↩︎ When a workflow should gain a bounded agent step“Specific: Clearly define what you want to achieve.”
↩︎ Standards for a reusable pattern library - 3.https://code.claude.com/docs/en/agent-sdk/overviewOfficial docs
“Control which tools run automatically, which need approval”
↩︎ When a workflow should gain a bounded agent step“Spawn specialized agents for focused subtasks”
↩︎ Deciding whether multi-agent has earned its cost“A hosted agent harness that runs the agent loop”
↩︎ Selecting the harness and framework“Package skills, agents, hooks, and MCP servers, and load them by local path”
↩︎ Standards for a reusable pattern library“Have Anthropic host the agent, configured through the Claude API | Managed Agents”
↩︎ Checkpoint - 4.
“Outside these situations, the coordination costs typically exceed the benefits.”
↩︎ Deciding whether multi-agent has earned its cost“Every additional agent represents another potential point of failure, another set of prompts to maintain, and another source of unexpected behavior.”
↩︎ Deciding whether multi-agent has earned its cost“Before adopting a multi-agent architecture, consider using the Tool Search Tool”
↩︎ Deciding whether multi-agent has earned its cost“While parallelism helps reduce total execution time compared to running all that work sequentially”
↩︎ Deciding whether multi-agent has earned its cost“These thresholds will shift as models improve. Current limits represent practical guidelines, not fundamental constraints.”
↩︎ Standards for a reusable pattern library“multi-agent systems often take longer overall than single-agent systems because of the sheer increase in total computation.”
↩︎ Exam trap 2“In our testing, multi-agent implementations typically use 3-10x more tokens than single-agent approaches for equivalent tasks.”
↩︎ Exam trap 3“An agent with too many tools (often 20+) struggles to select the appropriate one.”
↩︎ Exam trap 4“An agent with too many tools (often 20+) struggles to select the appropriate one.”
↩︎ Checkpoint“The primary benefit of parallelization is thoroughness, not speed.”
↩︎ Prediction“Before adopting a multi-agent architecture, consider using the Tool Search Tool, which lets Claude dynamically discover tools on-demand rather than loading all definitions upfront.”
↩︎ Checkpoint - 5.
“An agent harness (or scaffold) is the system that enables a model to act as an agent”
↩︎ Selecting the harness and framework“latency, token usage, cost per task, and error rates can be tracked on a static bank of tasks.”
↩︎ Standards for a reusable pattern library