CertSafari
    Anthropic Claude Certified Developer – Professional (CCDV-P)· Lessons

    Domain 2 · Lesson 5/25

    Resilience patterns for Claude API integrations

    Reliability Engineering for Claude Services

    19 min read
    5% of exam
    6 sources
    Published 19 Nov 2026
    Docs as of 18 Nov 2026

    What you will be able to do

    • Classify a failed or stopped Claude call as a rate limit, spend cap, overload, refusal, or context-window stop, and choose the recovery that fits it
    • Design retries that honour retry-after, back off exponentially and spread a fleet's retries with jitter
    • Explain when a circuit breaker and a degraded reply beat more retries during a sustained overload
    • Configure cross-model fallback for refusals, and plan cross-region and cross-platform fallbacks around what each platform actually supports

    Key concept

    Classify before you recover — Every failed or incomplete Claude call has a cause: a throttle, a spend cap, server overload, a safety refusal or a truncation. Each cause has its own correct recovery. If you apply one generic retry to all of them, you will wait on failures that can never succeed, put more load on an overloaded service, and miss refusals entirely.

    1.Three different kinds of "it didn't work"

    A production Claude integration fails in three different ways, and each one needs a different response. The first is an HTTP error: the request was throttled, rejected or not served. The second is a refusal: the request succeeded, but a safety classifier declined it. The third is a stop that truncates the output, such as model_context_window_exceeded. All three can look like "the call failed" to a caller, which is why one catch-all retry loop is the most common reliability bug in Claude integrations.

    Errors and stop reasons arrive through different channels. An error carries a non-200 status and a typed body, and the SDK raises it as an exception. A stop reason is a field on a successful response, so no exception handler ever sees it. Code that only wraps messages.create in try/except handles the first channel and is blind to the second.

    Error responses and the recovery each one calls for
    SignalWhat it meansRecovery
    429 rate_limit_error with retry-afterAn RPM, ITPM or OTPM limit was exceededWait the retry-after seconds, then back off
    429 rate_limit_error, no retry-after, error_code enforced_spend_limit_reachedThe usage tier's monthly spend cap was reachedDo not retry; it keeps failing until access resumes or the tier is raised
    400 invalid_request_error about specified usage limitsA spend limit you set yourself was reachedRaise or remove the limit
    500 api_errorAn unexpected internal errorRetry with exponential backoff; quote the request ID if it persists
    529 overloaded_errorThe API is temporarily overloadedBack off; if it is sustained, trip a circuit breaker and degrade
    504 timeout_errorThe request timed out while processingUse streaming for long requests rather than retrying the same call

    The spend-cap row is the one that catches teams out. It has the same rate_limit_error type as an ordinary throttle, but no retry-after header, and retrying cannot fix it. Branch on error.details.error_code to tell the two apart. A 429 can also appear while your dashboard shows usage below the limits: a sharp increase in traffic triggers ramp limits. Throughput planning for that case is covered in rate limits and ramp-up.

    Stop reasons that are not errors but still need handling
    stop_reasonWhat it meansRecovery
    refusalClaude declined to respondRead stop_details and retry on a fallback model
    max_tokensThe response reached your max_tokens limitRaise max_tokens or continue the response
    model_context_window_exceededThe response filled the model's context windowTreat the response as truncated
    pause_turnA server-tool loop reached its iteration limitSend the assistant content back to continue

    A context-window stop is not a throttle and not a refusal. Re-sending the request to escape a rate limit misreads it, and so does routing it through refusal fallback. The output is truncated, so the application must not present it as a complete answer.

    Checkpoint 1 of 9· Match them up

    A production integration runs into each of these signals. Match each one to the recovery it calls for.

    Tap a term, then the definition that fits it.

    Sources123

    2.Retries that do not recreate the burst

    The SDK already retries for you. Connection errors, rate limits and 5xx responses are retried with exponential backoff, twice by default, and the SDK honours retry-after when the header is present. max_retries tunes or disables this. Any retry loop you write sits on top of the SDK's, so three application retries on top of two SDK retries can mean many more attempts per user request than anyone intended.

    Rate limiting runs on a token bucket that refills continuously, not on a counter that resets at fixed intervals. So the problem is the shape of your traffic, not its total. A limit of 60 RPM can be enforced as one request per second, and a burst exceeds it even when the minute's total would fit. Fixed retry intervals keep a fleet synchronized: every worker that failed together retries together, and each retry round becomes a new burst.

    A retry policy for a fleet therefore has three parts:

    1. On a 429 with retry-after, wait that many seconds first. The server is telling you when capacity returns.
    2. Grow each later sleep exponentially, so a still-throttled worker backs off further instead of hammering at a constant rate.
    3. Add a random offset (jitter) to each worker's delay, so the fleet's retries spread across the refill window instead of landing together.

    Raising the retry cap or moving to a flat 5-second sleep fixes none of this, because the lockstep stays. Routing retries to a larger model to borrow its quota only moves the burst to another bucket.

    Timeouts need their own design. A gateway timeout shorter than a long generation turns every slow request into a timeout, then a retry, then another full generation, and the request still fails. For long outputs, stream. If you only want the finished message, the SDK can consume the stream and return the same object a non-streaming call would.

    Streaming a long request but collecting only the final messagepython
    client = anthropic.Anthropic()
    
    with client.messages.stream(
        max_tokens=128000,
        messages=[{"role": "user", "content": "Write a detailed analysis..."}],
        model="claude-sonnet-5",
    ) as stream:
        message = stream.get_final_message()
    
    print(next(block.text for block in message.content if block.type == "text"))

    Checkpoint 2 of 9· Check yourself

    A report-writing feature asks for very long outputs. The gateway in front of the app times out before generation finishes, so the app retries, and every retry times out the same way. What fixes this?

    Checkpoint 3 of 9· Exam question

    Birch Games runs a nightly enrichment job in production that tags player-support transcripts with Claude, using a fleet of 200 workers. At the top of each hour, a burst of 429 responses arrives, each carrying a retry-after value. The workers use a fixed 2-second retry loop. Traffic graphs show that loop re-creating the burst every 2 seconds, with each spike nearly as tall as the first. Request logs show almost all 200 workers sending within the same fraction of a second. The team already waits out the retry-after value on the first rejection. Output quality was validated on the current model, and company policy requires a formal re-evaluation before any model change. The job has a long overnight window, so extra delay per request is acceptable. What is the best fix?

    Sources32

    3.Circuit breakers for sustained overload

    Backoff is built for brief failures. A 529 overloaded_error means the API is temporarily overloaded, and during a provider-wide incident "temporarily" can mean minutes. In that window, every retry with backoff ties up a worker thread for the whole backoff schedule. Multiply that by the SDK's retries and your own, and the thread pool drains. Endpoints that never call Claude then start failing too, because nothing is left to serve them.

    A circuit breaker sits in front of the Claude client and tracks the recent error rate. While it is closed, calls pass through. When failures cross a threshold, it opens: calls fail immediately with a degraded reply, and no request leaves the service. After a cool-down it goes half-open and lets a few trial requests through. If they succeed it closes; if they fail it opens again. Failing fast protects the rest of the service, and it also stops your fleet from adding load to a provider that is already overloaded.

    Other fixes look reasonable but make things worse. A larger thread pool and a longer timeout just let more requests wait on the same outage. More retries add load to an overloaded service and push latency higher. Restarting pods on every 529 treats a provider-side condition as a local fault and throws away healthy capacity. When the breaker is open, lower max_retries on the SDK client as well, so its built-in retries do not quietly rebuild the waiting you just removed.

    Checkpoint 4 of 9· Check yourself

    During a provider-wide incident, a sustained run of 529 overloaded_error responses hits a team's service. They have already removed their own application-level retry loop, yet each user request still holds a worker thread through several backed-off attempts before it fails. What explains this, and what should they change?

    Checkpoint 5 of 9· Exam question

    Granite Insurance runs a claims-assistant chat backend on a shared worker thread pool that also serves its policy-lookup and claims-status endpoints. During a provider-wide incident, its Claude calls return 529 responses. Each request retries 3 times with backoff, and traces show nearly every call still fails after the final attempt, so threads stay occupied for the whole retry window. p99 latency has reached 90 seconds, and the exhausted pool now takes down the unrelated endpoints. The provider's status page gives no recovery estimate. Product owners accept a short, simplified reply in chat during outages. On-call engineers want normal behavior to resume automatically once Claude recovers, without manual intervention or a full traffic surge at the first sign of recovery. What is the best change?

    Sources3

    4.Degrading gracefully instead of failing whole

    Graceful degradation means deciding in advance which parts of a feature are essential and which are optional. Take contract review. Clause extraction is the core output and a risk narrative is an extra. If one sustained overload fails both and the user sees an error page, the design has treated the optional step as mandatory.

    Honest degraded replies include:

    • Returning the core output that did succeed, with a notice that the optional part is delayed, and queuing that part for background retry.
    • Serving the most recent cached result for unchanged inputs, labelled as stale with its timestamp.
    • Answering from a fallback model and recording that it did so, so quality can be checked later.

    The label is what keeps these honest. Unlabelled stale data or silent substitutes become correctness bugs.

    The tempting wrong options pretend analysis happened when it didn't. Auto-approving contracts that have no analysis turns an availability problem into a compliance problem. A keyword template that declares "no material risks found" is a fabricated result shown with the same confidence as a real one. A generic error page is honest but throws away the clause extraction the user could have had. The same rule covers partial model output: a response interrupted mid-stream is incomplete, and it should never be shown as an answer.

    Checkpoint 6 of 9· Check yourself

    Clause extraction for a contract has come back complete. The optional risk-narrative call streamed half a narrative, then stopped with stop_reason refusal. What is the honest degraded reply to the user?

    Checkpoint 7 of 9· Exam question

    Heron Labs runs contract-review software for its in-house legal customers. Each upload triggers clause extraction, which is the core deliverable, followed by an optional risk narrative that reviewers read when time allows. During a sustained period of 529 overloaded responses, both steps fail and users see an error page. Reviewers have said clause extraction alone lets them start work, and many re-open documents that have not been edited since their last analysis. Counsel has ruled that no contract may advance in the approval workflow, or be described as low risk, without output the system actually produced from that document. Support also reports that users repeatedly hit retry on the error page. What should the team do to degrade gracefully? Select TWO.(Select 2)

    0 of 2 selected

    Sources4

    5.Cross-model fallback for refusals

    Classifier refusals arrive as normal responses. The status is 200, stop_reason is "refusal", content is usually empty, and stop_details.category names the policy area: cyber, bio, frontier_llm, reasoning_extraction or general_harms. The usual recovery is to send the same request to another model. Retrying the same model is the wrong move, because it usually refuses again. Branch on stop_reason itself, not on category or explanation; both of those can be null, and the explanation text is not stable.

    There are three ways to set up fallback, and which ones you can use depends on platform and model. Server-side fallback (beta) is the simplest: set fallbacks to "default" with the beta header, and the API retries on the model Anthropic recommends for that refusal category, all inside one call. SDK middleware works on any platform and walks your fallback list in order. A manual retry gives full control, and you redeem fallback credit so you don't pay the prompt-cache write twice. For multi-turn conversations, stay on the model that accepted rather than switching back.

    Server-side fallback using the default routingpython
    client = Anthropic()
    
    response = client.beta.messages.create(
        model="claude-fable-5",
        max_tokens=1024,
        messages=[{"role": "user", "content": "Hello, Claude"}],
        fallbacks="default",
        betas=["server-side-fallback-2026-07-01"],
    )
    print(response.model)

    Fallback has several edge cases:

    • reasoning_extraction has no recommended fallback model, because the prompt itself is the problem. Ask for a short explanation instead of the model's internal reasoning.
    • Claude Haiku 5.5 has no server-side fallback and no fallback credit.
    • The fallbacks parameter does not carry over to model calls made from inside tool execution, so subagents need their own.
    • Every request path needs fallback configured, including retry handlers and background workers.
    • A refusal still counts against your rate limits, so a refusal storm also uses up throughput.

    Fallback paths also need their own monitoring. Emit one event per refusal and one per fallback-served response (a fallback_message entry in usage.iterations marks the latter), and alert on the gap between the two counts. Track output quality on the fallback path separately too, because high availability through a weaker model can hide an accuracy drop that nobody sees. Building those signals into dashboards is covered in production monitoring and alerting.

    Checkpoint 8 of 9· Fill the gap

    This call should send a refused request to Anthropic's recommended fallback model within the same API call. Which parameter name completes it?

    client = Anthropic()
    
    response = client.beta.messages.create(
        model="claude-fable-5",
        max_tokens=1024,
        messages=[{"role": "user", "content": "Hello, Claude"}],
         ? ="default",
        betas=["server-side-fallback-2026-07-01"],
    )
    print(response.model)

    Sources4

    6.Cross-region and cross-platform fallback, and feature parity

    Cross-region fallback is often handled by choosing the right endpoint type rather than writing your own failover. On Amazon Bedrock, global endpoints route requests dynamically to regions with available capacity and carry no premium. Regional endpoints pin traffic to a geography, which data-residency and compliance requirements may demand, and they cost 10 percent more. If residency is flexible, the global endpoint gives you regional failover for free. If it is not, any fallback has to stay inside the permitted regions.

    Cross-platform fallback, such as sending traffic from the Claude API to a cloud provider during an incident, only works for features that exist on both sides. The request formats differ: on Google Cloud's Agent Platform the model goes in the endpoint URL, not the body, and model availability varies by region. Limits differ too: Bedrock caps request payloads at 20 MB, against 128 MB for the first-party Messages API. Model line-ups differ, and lifecycle dates on partner platforms are set by the partner. A fallback target that lacks the model, the payload size or the feature your request depends on fails exactly when you need it most.

    Refusal-fallback support and listed models by platform
    PlatformServer-side fallbackModels listed
    Claude APIYes (beta)Fable 5.2, 5.1, 5; Mythos 5.2, 5.1; Opus 5.5, 5; Sonnet 5.5; Haiku 5.5
    Claude Models on Google Cloud (Anthropic managed)Yes (beta)Fable 5.2, 5.1, 5; Mythos 5.2; Opus 5.5, 5; Sonnet 5.5; Haiku 5.5
    Claude Platform on AWSNo; use SDK middleware or manual retryFable 5.2, 5.1, 5; Opus 5.5, 5; Sonnet 5.5; Haiku 5.5
    Amazon BedrockNo; use SDK middleware or manual retryFable 5.1, 5; Mythos 5.1; Opus 5.5, 5; Sonnet 5.5; Haiku 5.5
    Microsoft Foundry (hosted on Azure)No; use SDK middleware or manual retryOpus 5.5, 5; Sonnet 5.5; Haiku 5.5

    Batches follow different rules again. A batch request that sets fallbacks comes back as an errored item. A refused batch item comes back as succeeded with stop_reason: "refusal", so a batch pipeline has to scan its results for refusals and resubmit them on a fallback model. Treat parity as something you test: for each fallback target, confirm that the model, the parameters and the request size your primary path uses are all accepted there.

    Checkpoint 9 of 9· Check yourself

    Your app runs Claude Opus 5.5 on the Claude API, and server-side fallback (fallbacks="default") handles its refusals. During an incident, the failover plan moves the same request code to Amazon Bedrock. What does the Bedrock path need so that refused requests still reach a fallback model?

    Sources564

    Sources

    (6)

    Every claim above is drawn from one of these pages, quoted as it was written on the date shown.

    1. 1.
      “Unlike errors, which indicate failures in processing your request, stop_reason tells you why Claude completed its response generation.”
      ↩︎ Three different kinds of "it didn't work"
      “Check this field to decide whether to use the response as-is, continue the conversation, retry, or fall back to another model.”
      ↩︎ Key concept
    2. 2.
      “Use it to tell this response apart from a rate limit.”
      ↩︎ Three different kinds of "it didn't work"
      “your capacity is continuously replenished up to your maximum limit, rather than being reset at fixed intervals.”
      ↩︎ Retries that do not recreate the burst
      “When the response has a retry-after header, wait that many seconds before retrying.”
      ↩︎ Retries that do not recreate the burst
      “The error type is rate_limit_error, the same as for a rate limit, but the response has no retry-after header.”
      ↩︎ Checkpoint
      “Short bursts of requests can exceed the limit and trigger rate limit errors.”
      ↩︎ Prediction
    3. 3.
      “A sharp increase in usage can also cause a 429 even while you’re under your rate limits”
      ↩︎ Three different kinds of "it didn't work"
      “with exponential backoff, twice by default, honoring the retry-after header when present”
      ↩︎ Retries that do not recreate the burst
      “For long-running Messages API requests, use the streaming Messages API.”
      ↩︎ Retries that do not recreate the burst
      “529 - overloaded_error: The API is temporarily overloaded.”
      ↩︎ Circuit breakers for sustained overload
      “The SDK client accepts max_retries to configure or disable this behavior.”
      ↩︎ Circuit breakers for sustained overload
      “A tier spend-cap 429 has no retry-after header and keeps failing until access resumes”
      ↩︎ Exam trap 1
      “The official SDK automatically retries transient failures (such as connection errors, rate limits, and 5xx server errors) with exponential backoff, twice by default”
      ↩︎ Checkpoint
    4. 4.
      “A refusal can arrive before any output, or mid-stream after partial output. In either case, treat any partial output as incomplete and discard it.”
      ↩︎ Degrading gracefully instead of failing whole
      “The fallbacks parameter does not carry over to model calls made from inside tool execution.”
      ↩︎ Cross-model fallback for refusals
      “This category has no recommended fallback model, so change the prompt rather than retry the request.”
      ↩︎ Cross-model fallback for refusals
      “For multi-turn conversations, keep using the fallback model for subsequent turns rather than switching back.”
      ↩︎ Cross-model fallback for refusals
      “The request still counts against your rate limits.”
      ↩︎ Cross-model fallback for refusals
      “Server-side fallback is not available for batches”
      ↩︎ Cross-region and cross-platform fallback, and feature parity
      “Re-sending a refused request to the same model usually earns another refusal.”
      ↩︎ Exam trap 2
      “Instrument refusals as their own signal.”
      ↩︎ Exam trap 3
      “Claude Haiku 5.5 has no server-side fallback, so for it use client-side fallback with the SDK middleware or write the retry yourself.”
      ↩︎ Exam trap 4
      “A refusal is an HTTP 200, so monitoring built on error rates or 5xx responses never sees it.”
      ↩︎ Prediction
      “The simplest setup, in beta on the Claude API and on Claude Models on Google Cloud (Anthropic managed): set fallbacks to "default"”
      ↩︎ Checkpoint
    5. 5.
      “Dynamically route requests to regions with available capacity”
      ↩︎ Cross-region and cross-platform fallback, and feature parity
      “Are required for data residency and compliance requirements”
      ↩︎ Cross-region and cross-platform fallback, and feature parity
      “Bedrock limits request payloads to 20 MB.”
      ↩︎ Cross-region and cross-platform fallback, and feature parity
    6. 6.
      “Note that Anthropic model availability varies by region.”
      ↩︎ Cross-region and cross-platform fallback, and feature parity
      “Lifecycle dates on partner-operated platforms are set by the partner and can differ from the Claude API schedule.”
      ↩︎ Cross-region and cross-platform fallback, and feature parity

    Ready to test yourself?

    Practise Anthropic Claude Certified Developer – Professional (CCDV-P) in quiz mode.

    Spotted a mistake, or was something unclear? Tell us.