What you will be able to do
- Use a small model with structured output as an input or output screen, and treat a refusal from the screen as a harmful verdict
- Place PII redaction where it keeps sensitive values out of the model context, transcripts and replies
- Recognize a classifier refusal, discard partial output, and choose a fallback that does not bypass your own checks
- Tune a classifier's false-positive and false-negative rates per category from labeled production and adversarial data
1.Classifier screens on input and output
Content guardrails inspect what flows through an agent rather than what it does. That means the user's message on the way in, the reply on the way out, and the data tools return. These guardrails catch problems that action controls never see. A reply that discloses a customer's details is not a tool call, so no permission rule ever evaluates it.
The standard screen is a lightweight model such as Claude Haiku 5.5 that classifies user input before it reaches the main conversation. Constrain its response with structured outputs so your code receives a boolean, not prose to parse. The same pattern works on the way out: screen the agent's reply before it is displayed. For input, add validation against known injection phrasing; an LLM given examples of jailbreak language can act as a generalized validator.
{ "output_config": { "format": { "type": "json_schema", "schema": { "type": "object", "properties": { "is_harmful": { "type": "boolean" } }, "required": ["is_harmful"], "additionalProperties": false } } } }The screen itself can refuse. Haiku 5.5 runs safety classifiers that can decline the screening request, and the response then carries stop_reason: "refusal" with no verdict field. A pipeline that treats this as an error and lets the input through fails open on exactly the inputs most likely to be harmful. Treat the refusal as a harmful verdict. When the same user triggers the same block repeatedly, escalate: tell them the behavior violates usage policy, and throttle or ban them. Isolating injected content in tool results is a separate defense, covered in threat defense for agentic systems.
Checkpoint 1 of 6· Fill the gap
This structured-output schema constrains a Haiku harmlessness screen. Which type completes it so that your pipeline gets a verdict it can branch on directly, with no free text to parse?
{ "output_config": { "format": { "type": "json_schema", "schema": { "type": "object", "properties": { "is_harmful": { "type": " ? " } }, "required": ["is_harmful"], "additionalProperties": false } } } }The screen is meant to return one true/false verdict. Typing is_harmful as boolean means your code receives a boolean rather than prose or labels it has to interpret. A string would allow free text, and an object or integer would add structure the pipeline then has to decode.
Source: platform.claude.comSources1
2.Redacting PII before it reaches the model
Where you redact decides what is protected. PII that a tool returns enters the model's context, then the stored transcript, and sometimes a reply. A filter at the end of that chain protects only the reply. A filter at the start protects all three.
So redact at the tool boundary. The application code that returns a CRM record replaces SSNs and phone numbers with placeholder tokens and keeps the token-to-value map itself. The model can still reason about "the customer's SSN" without ever holding the digits. Store transcripts with the tokens in place, and restore real values only when the output is delivered to an authorized recipient. This is least privilege applied to data: don’t give Claude access to secrets it doesn’t need. The secure deployment pattern for credentials works the same way. A proxy outside the agent holds the real value, and the agent never sees the actual credentials. Telemetry is another copy of the data, so if exported metadata can contain sensitive values, filter or redact them at your collector.
The common alternatives fail at specific points. A system-prompt rule telling Claude not to write SSNs leaves them in the context and the transcript, and depends on the model complying. A regex filter on assistant output cleans replies and tool-call arguments but leaves the raw values in context and in stored transcripts. Encrypting the transcript store protects it at rest, but the tool results the model reads still contain the raw values.
Checkpoint 2 of 6· Check yourself
A support agent calls a CRM lookup tool that returns customer SSNs. Compliance requires that the raw SSNs never appear in the model's context or in stored transcripts, while the agent can still refer to them. Which design meets that requirement?
Only redaction at the tool boundary keeps the raw values out of all three places: context, transcript and reply. The model reasons over tokens and never holds the digits. An output filter cleans only replies and arguments, encryption protects only data at rest, and a prompt rule leaves the values in context and relies on the model complying.
“don’t give Claude access to secrets it doesn’t need”Source: platform.claude.com
Checkpoint 3 of 6· Exam question
Indigo Health runs a production claims agent that calls an internal CRM tool, and every lookup returns the full customer record. In an audit, Compliance finds Social Security numbers and phone numbers in the model context, in the stored conversation transcripts, and only occasionally in replies shown to customers. Reply leaks are rare. The system prompt already tells Claude to avoid quoting identifiers, yet raw values still appear in transcripts that engineers and analysts read while debugging. The transcript store is already encrypted at rest, and the CRM's response format cannot be changed. Claims handlers need a real phone number only when a callback is actually placed. Select TWO changes that best reduce exposure.(Select 2)
Correct answers: B, D — Swap SSNs and phone numbers in CRM tool output for placeholder tokens before the model sees them, with the mapping held in app code.; Store transcripts with tokens in place of PII, restoring real values only at the point of delivery to authorized recipients.
- A. A prompt rule is a request, not an enforced control. The full CRM records still enter the context and are still written to transcripts, and the model can still repeat values despite the instruction.
- B. Correct. Tokenizing PII in tool results before they reach the model means raw SSNs and phone numbers never enter its context, so it cannot echo them. The token-to-value mapping stays in application code.
- C. Filtering assistant output addresses only what the customer sees. The raw CRM data still sits in the model context and in the stored transcripts, which is where compliance found most of the exposure.
- D. Correct. Transcripts written with tokens hold no raw PII, which closes the logging exposure. Real values come back only in a controlled step at delivery, for recipients cleared to see them.
- E. Encryption and tighter read access protect the stored transcripts but leave the raw values in them. Because tool results pass through unchanged, the model still receives the PII and can still repeat it in replies.
3.Refusal handling and safety fallbacks
A classifier refusal from Claude is a successful HTTP 200 response, not an error. It carries stop_reason: "refusal", and stop_details.category names the policy area. The explanation text is not stable, so display it rather than parse it. A refusal can arrive before any output or partway through a stream. In both cases, discard any partial output as incomplete.
| category | Triggered by | Billed before any output |
|---|---|---|
| cyber | Requests that could enable cyber harm; benign security work can also trigger it | No |
| bio | Requests that could enable biological harm | Yes |
| reasoning_extraction | Prompts asking the model to reproduce its internal reasoning in the response | Yes |
| general_harms | Usage-policy areas outside the named categories | No |
The usual safety fallback is to retry the same request on another Claude model. Server-side fallback, in beta, does this inside a single API call. Set fallbacks to "default" and the API reruns a declined request on the model Anthropic recommends for that refusal category. For categories with no recommended fallback, the refusal stands. Claude Haiku 5.5 has no server-side fallback, so for Haiku use the SDK middleware or write the retry yourself. A reasoning_extraction refusal has no recommended fallback either. Change the prompt instead of retrying.
client = Anthropic()
response = client.beta.messages.create(
model="claude-fable-5",
max_tokens=1024,
messages=[{"role": "user", "content": "Hello, Claude"}],
fallbacks="default",
betas=["server-side-fallback-2026-07-01"],
)
print(response.model)A model refusal and a block by your own guardrail are different events. Retrying a refusal on a fallback model is reasonable when the request is legitimate for your application. A reply that your output screen blocked is a different case. Regenerating it on a larger model and sending the result straight to the customer means the new reply skips the check that caught the first one. Any replacement reply must pass the same screen before it is delivered.
Checkpoint 4 of 6· Check yourself
Your support agent runs on Claude Haiku 5.5. To keep users from hitting dead ends, you set fallbacks to "default" with the beta header. A legitimate customer request then comes back with stop_reason "refusal". What should you expect, and what is the right fix?
Server-side fallback does not cover Claude Haiku 5.5, so setting fallbacks to "default" gives it no automatic retry. Use client-side fallback through the SDK middleware or write the retry yourself. A refusal is also a successful HTTP 200 response, not an error, and the explanation text is not stable enough to parse.
“Claude Haiku 5.5 has no server-side fallback, so for it use client-side fallback with the SDK middleware or write the retry yourself.”Source: platform.claude.com
Sources4
4.Tuning false positives and false negatives
Every classifier makes two kinds of error, and production makes only one of them visible. False positives, where benign replies are blocked, show up as complaints and entries in the block log. False negatives, where harmful content gets through, show up only when someone goes looking for them, usually a red team.
Error rates differ by category, even for Anthropic's own classifiers. Benign cybersecurity work can trigger the cyber category, and the categories billed before any output are the ones where Anthropic measures low volumes of false positives. One global threshold therefore tunes every category with the same setting, even though their error rates differ.
Consider a worked case. An output screen blocks 9% of a support agent's replies. Most of those blocks are benign refund-policy answers, yet a red-team run still leaks data. Raising a single global threshold cuts benign blocks by letting more leaks through. Exempting refund topics creates a gap that an attacker can exploit by phrasing a request as a refund question. The workable step is measurement. Build a labeled set from production blocks, which show the false positives, and adversarial misses, which show the false negatives. Then tune rubrics and thresholds per category and score each one on precision and recall. The broader evaluation approach is covered in evaluation system design.
Block logs contain only blocked items, so they measure false positives. The harmful replies that passed never appear there. Only adversarial test cases reveal those false negatives, and a threshold change has to be scored against both.
Checkpoint 5 of 6· Check yourself
An output screen on a developer-help agent uses one global threshold. Users complain that benign explanations of security bugs keep getting blocked, and a red-team run shows customer data still leaking in another category. What should the team do next?
Error rates differ by category, and a block log shows only false positives. Raising a global threshold lets more leaks through, and a topic exemption creates a gap attackers can exploit. A labeled set covering both error types, tuned and scored per category, addresses both problems.
“These are the categories where Anthropic measures low volumes of false positives, as of September 2026.”Source: platform.claude.com
Checkpoint 6 of 6· Exam question
Kestrel Labs runs a customer support agent for its subscription analytics platform. A Haiku-based output classifier screens every reply before delivery and currently blocks 9% of them. A review of the blocked replies shows 70% are benign refund-policy answers, and support leads report customers abandoning chats after receiving fallback messages. Kestrel's security team also ran a red-team set of 200 adversarial attempts, and two still produced replies that leaked customer data. The classifier applies one rubric and one block threshold to every kind of reply, and no one has recorded which reply categories the false blocks and the misses fall into. Compliance requires that any change be justified with measured evidence, and the team wants to keep response latency low. What is the best next step?
Correct answer: B — Assemble a labeled set from production blocks and adversarial misses, then tune per-category rubrics and thresholds on precision and recall.
- A. One global threshold moves false positives and misses in opposite directions. Loosening it lowers the benign block rate, but the classifier already let two of 200 attacks through, so more data would leak. Watching production afterward only detects the harm.
- B. Correct. A labeled set drawn from real blocks and adversarial failures lets precision and recall be measured per category. Refund-policy blocks can then be relaxed while the category that leaked data is tightened, based on evidence.
- C. Exempting a topic removes enforcement from replies that can still contain customer data, and a prompt instruction is not a reliable control. It also does nothing about the two leaks that already got through, so the classifier's weakness stays.
- D. Regenerating blocked replies and sending them unchecked removes the guardrail for exactly the replies it flagged. Stronger prompt instructions do not guarantee the new reply is clean, and the extra model call adds latency and cost. The detection gaps stay unmeasured.
Sources4
Sources
(4)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/mitigate-jailbreaksOfficial docs
“Use a lightweight model such as Claude Haiku 5.5 to pre-screen user input before it reaches your main conversation.”
↩︎ Classifier screens on input and output“consider throttling or banning users who repeatedly attempt to circumvent your application’s guardrails”
↩︎ Classifier screens on input and output“don’t give Claude access to secrets it doesn’t need”
↩︎ Redacting PII before it reaches the model“treat a response with stop_reason: "refusal" as a harmful verdict”
↩︎ Exam trap 1“treat a response with stop_reason: "refusal" as a harmful verdict”
↩︎ Prediction - 2.
“The agent never sees the actual credentials”
↩︎ Redacting PII before it reaches the model - 3.
“If these can contain PHI, filter or redact them at your collector.”
↩︎ Redacting PII before it reaches the model - 4.
“In either case, treat any partial output as incomplete and discard it.”
↩︎ Refusal handling and safety fallbacks“Claude Haiku 5.5 has no server-side fallback, so for it use client-side fallback with the SDK middleware or write the retry yourself.”
↩︎ Refusal handling and safety fallbacks“Benign cybersecurity work can also trigger this category.”
↩︎ Tuning false positives and false negatives“These are the categories where Anthropic measures low volumes of false positives”
↩︎ Tuning false positives and false negatives“Claude Haiku 5.5 has no server-side fallback, so for it use client-side fallback with the SDK middleware or write the retry yourself.”
↩︎ Exam trap 2“These are the categories where Anthropic measures low volumes of false positives, as of September 2026.”
↩︎ Checkpoint