Retry with Error Feedback in Extraction Pipelines

1.Schema syntax errors vs semantic validation errors

Before you design a retry loop, work out which failures actually need one. The structured outputs documentation lists what goes wrong when nothing constrains the output: parsing errors from invalid JSON, missing required fields, inconsistent data types, and schema violations that need error handling and retries. Its summary is blunt: without structured outputs, Claude can produce malformed JSON or invalid tool inputs that break your application.

There are two features that remove this whole category. JSON outputs (output_config.format) and strict tool use (strict: true) both use constrained decoding, so the model can only produce tokens that are valid under the schema. The strict tool use page says what that means for your code: functions get correctly typed arguments every time, and there is no need to validate and retry tool calls.

This is where people go wrong. Constrained decoding guarantees shape, not meaning. A schema can require stated_total to be a number. It can't require that number to equal the sum of the line items. A schema with both a patient and a guardian object can be satisfied by a date of birth placed under the wrong one, as long as both objects allow that field. These are semantic validation errors: the output parses, the types are right, and it's still wrong. Only your own validation logic catches them, and they're what a retry loop is for.

Sources12

2.Retry with specific error feedback

Once your validator finds a semantic error, the weakest response is to send the identical request again and hope for a different result. Nothing in that request tells the model what it got wrong. The better pattern is the one the tool-use tutorial teaches for failing tools: send the error back instead of crashing, with is_error: true on the tool result. In the tutorial's words, Claude reads the error and can retry with corrected input, ask for clarification, or explain the limitation.

The content moderation cookbook shows this as an extraction-quality loop. A static validator checks Claude's compiled rules, and any problems it finds (an invented field, an enum value outside its allowed set) go back to Claude as a fix-only turn. The cookbook says these exact messages go back to Claude to fix. Look at how much one of them tells the model:

  "bad2": [
    "bad2.when: 'liquor' not in enum for 'ad_category': ['alcohol', 'gambling', 'financial_services', 'pharma_supplements', 'weight_loss', 'cosmetics_skincare', 'apparel_footwear', 'electronics', 'food_beverage', 'other']"
  ],

A message like "extraction invalid" leaves the model guessing. This one says where the problem is (bad2.when), what was wrong ('liquor'), and what would be accepted. The evaluator-optimizer pattern explains why this works: a feedback loop fits when LLM responses can be measurably improved by feedback and the evaluation criteria are clear. The cookbook's guidance on writing feedback also fits validation errors: ask for a one-line scoreboard, then one bullet per failure saying what's wrong and what to do. In an extraction pipeline, that means each failed check becomes one line naming the field path, the problem, and the expected relationship, such as "line_items sum to 1,150 but stated_total is 1,200".

Sources3456

3.Building the self-correction follow-up request

A retry is a new API call, and the Messages API keeps no memory between calls. The documentation says it directly: the API is stateless, so you always send the full conversation history. If your follow-up contains only the error message, Claude has nothing to fix. If it contains only the failed JSON and the error, Claude can rearrange keys but can't check the values against anything.

A follow-up that leads to real self-correction contains three things:

1. The original document, with the original extraction instructions. Without the source, the model can only shuffle what it already produced. With it, the model can re-read the form and confirm each value. 2. The failed extraction, as an assistant turn. The docs say earlier turns don't have to come from Claude and that you can use synthetic assistant messages. So you can rebuild the conversation as if the failed output had just been produced, even in a fresh request. 3. The specific validation errors, in a final user turn that asks for a complete corrected extraction checked against the document.

The hallucination guide calls this iterative refinement: using Claude's outputs as inputs for follow-up prompts, asking it to verify or expand on what it said before. The multi-turn sample below has exactly this structure: user, assistant, user.

message = anthropic.Anthropic().messages.create(
    model="claude-opus-5-5",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Hello, Claude"},
        {"role": " ? ", "content": "Hello!"},
        {"role": "user", "content": "Can you describe LLMs to me?"},
    ],
)
print(message)

Sources78

4.Self-correction fields: calculated_total and conflict_detected

A validator can only compare what it can see, so design the schema to expose the comparison. Instead of a single total, extract two fields: stated_total, the figure the document prints, and calculated_total, the figure the model computes from the line items it extracted. Your code then compares the two. A mismatch is a precise, checkable semantic error, and it becomes a one-line retry message such as "calculated_total is 1,150 but stated_total is 1,200". The model can re-read the source and fix a misread digit or a missed line item. This is the kind of document-versus-data check the cookbooks describe for invoice processing: extract data from documents, then flag discrepancies.

Sometimes the document itself is the problem. The printed total really doesn't match its own line items, or two sections of the source give different values. No retry can fix that, because every correct re-reading produces the same disagreement. For this case add a conflict_detected boolean. When the source is internally inconsistent, the model sets it to true and reports both values instead of silently picking one. The pipeline then routes the record to human review rather than looping. This follows the same logic as the limit of retry: a retry helps only when the right answer is in the document. It also follows the guidance to give Claude permission to admit uncertainty, and to make a field optional when the source may not contain what the schema asks for.

In short, calculated_total next to stated_total turns a hidden semantic error into a flagged discrepancy, and conflict_detected separates an extraction mistake, which a retry can correct, from a contradiction in the source, which needs a person.

Sources8910

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.Once strict tool use or structured outputs is on, extraction output no longer needs to be validated.Why is that wrong?

   The guarantee covers schema compliance: syntax, types, and required fields. Values that don't add up, or that sit in the wrong schema-valid object, still pass and need a semantic validator.

   Covered in Schema syntax errors vs semantic validation errors

2. 2.A retry only needs the validation error, because the model remembers the document from the first call.Why is that wrong?

   The API is stateless. Each follow-up has to resend the original document and the failed extraction along with the errors, or the model has nothing to check its correction against.

   Covered in Building the self-correction follow-up request

3. 3.When calculated_total and stated_total disagree, keep retrying until they match.Why is that wrong?

   A retry helps only when the correct answer is in the document. If the source contradicts itself, set conflict_detected and escalate instead of looping.

   Covered in Self-correction fields: calculated_total and conflict_detected