Effort and Model Choice: Trading Accuracy for Latency and Cost

1.Two kinds of lever: free wins and real trade-offs

When you configure a Claude integration, you choose the model, the effort level and the architecture for each request. Anthropic's cost guidance sorts these choices into two groups. Free wins lower spend without touching quality: prompt caching, token hygiene, a prompt audit against the model you actually run, batch processing, and workspace spend limits. Trade-offs buy speed or savings with intelligence: model choice, effort, output caps and task budgets, an elapsed-time clock, and multi-model architectures.

This split decides the order in which you justify a configuration. Take the free wins first, because they move the workload toward the frontier at no cost to accuracy. Prompt caching was the largest lever Anthropic measured. Every turn of an agent loop resends the whole growing conversation, and caching bills that repeated prefix at a tenth of the input price. Only after the free wins are in place should you decide where on the accuracy-cost-latency curve the workload belongs.

One 'free' lever does cost something on the latency axis. Batch processing is 50% off, but only for work that can wait up to 24 hours. It suits overnight evaluation runs and is wrong for anything a user is waiting on.

Sources1

2.Effort: the main accuracy-latency dial within a single model

The effort parameter sets how many tokens Claude spends on a response, so one model can be tuned to be more thorough or more economical. You set it with output_config.effort. It applies to every output token: text, tool calls and their arguments, and thinking when it is active. That is why it works whether or not thinking is enabled, and why lower effort also means fewer and terser tool calls. Defaults vary by model. Most models default to high, but Claude Opus 5.5 defaults to medium.

response = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=4096,
    messages=[
        {
            "role": "user",
            "content": "Analyze the trade-offs between microservices and monolithic architectures",
        }
    ],
     ? ={"effort": "medium"},
)

Effort changes the bill in proportion to everything the model does, but it only improves accuracy for the part of that work your task actually needs. Whether a given step up in effort pays depends on the workload, and the measurements below show both shapes.

One latency caveat matters when you choose effort for speed. Effort controls thinking volume, not visible response length. On Claude Opus 5, changing effort does not reliably shorten responses, so if you need shorter visible output you should prompt for length instead of relying on a lower effort level.

The takeaway is to measure your own curve. You cannot tell from how hard a task sounds which shape its curve will have: research work was nearly flat, while long-horizon coding is where effort genuinely buys accuracy, and there lowering effort is a real trade-off.

The per-model guidance follows the same logic. On Claude Fable 5.1, step down to medium or low for routine or latency-sensitive work once your evals show quality holds. On Claude Sonnet 5.5, start chat and other latency-sensitive work at medium or low. If effort has to stay low for latency on a model that then reasons too shallowly, the Opus 4.7 guidance is to add targeted guidance such as telling the model the task involves multistep reasoning.

Sources21

3.Justifying a setting: sweeps, cost per solved task, and re-running failures

Since the task description won't tell you which kind of curve you have, you justify a configuration with measurements, not intuition. Sweep two or three effort levels on a sample of your own traffic and read the answer off the curve. Run each level in a separate session, because changing top-level effort mid-session invalidates the prompt cache and distorts the comparison. That single-model curve is also the baseline any multi-model design has to beat. In Anthropic's measurements, a multi-model setup that looked cheaper than the default single model cost more than the same model at lower effort.

When comparing models, use cost per completed task, not cost per token. The same text costs about 30% more tokens on Claude Opus 4.7 and later, so a per-token comparison makes newer models look more expensive by construction. Model choice guidance agrees that tuning effort is often a better lever than switching models. The decision table gives the order of moves: if costs are too high and quality is fine, sweep effort down. If quality is not good enough, restore effort if you lowered it, and otherwise try the next tier up at low effort.

Two multi-model strategies pair a lower-cost model with a frontier model, and each pays off in a different situation. An advisor setup suits a lower-cost executor that stalls only on hard decisions. It pays off when the advisor is priced well above the executor and is actually consulted, so first price the advisor's model alone at low effort and measure the consult rate. An orchestrator setup suits work that exceeds one context window, where you delegate partitions to cheaper workers. Whichever you pick, justify it against the single-model effort curve.

If the outcome can be checked by tests or a verifier, the cheapest policy is not a fixed effort level at all. Run every task at low and re-run only the failures at high. On the SWE-bench Pro subset with Claude Opus 5.5, about 97% of tasks passed for about $0.17 each, against 95.3% for $0.29 when everything ran at high. This policy depends on a reliable failure signal, since a checker that passes bad work lets those failures through. It also costs latency: every first-pass failure takes two runs of wall-clock time. That makes it a good fit for offline or asynchronous work, and a poor fit when a user is waiting on every task.

Sources13

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.Hard tasks always need high or max effort to be accurate.Why is that wrong?

   Whether effort improves accuracy depends on the workload. On some demanding research benchmarks accuracy stayed flat across effort while cost rose, so you have to measure the curve.

   Covered in Effort: the main accuracy-latency dial within a single model

2. 2.When choosing between models, compare their per-token prices.Why is that wrong?

   Newer models can use more tokens for the same text, and the deciding factor is how many tasks get solved. Compare cost per solved task.

   Covered in Justifying a setting: sweeps, cost per solved task, and re-running failures