What you will be able to do
- Explain why a cached prefix still limits the context window, and why that matters when choosing a context policy
- Choose between on-demand, background, threshold and self-written compaction for a given latency and recall need
- Configure tool-result and thinking-block clearing without destroying prompt-cache hit rates
- Use the memory tool to keep durable knowledge outside the active context
- Change instructions or tool availability mid-conversation without invalidating the cached prefix
- Keep a large tool catalog out of the window with tool search and defer_loading
Key concept
Window occupancy vs. cache price — Prompt caching makes repeated tokens cheaper but does not remove them from the context window. A context policy has to do two things: decide which tokens stay in the window, and avoid rewriting the cached prefix so often that the savings disappear.
1.Why a long-running agent needs a context policy
Long-running agents hit two limits as they work: the size of the context window and the cost ceiling. An incident-triage agent that reads log files, or a coding agent that is 200 turns in, adds tokens on every turn. Most of those tokens are tool output that mattered once and then went stale. Size is not the only problem. Response quality drops as a conversation grows, and irrelevant content degrades model focus. Context is a finite resource with diminishing returns, so you curate it. You don't just let it fill up.
Prompt caching does not get you out of this. A cached prefix is cheaper to read, but it still occupies the context window. A team whose 150k-token prefix is fully cached cannot treat those tokens as free space. If they add 900k tokens of documents to a 1M-token window, the request exceeds the window. If the input alone exceeds the window, the API returns a 400 invalid_request_error ("prompt is too long"). The token counting API lets you check this before you send.
That leaves two separate questions for every policy below. Does it shrink what sits in the window? What does it do to the cached prefix? Summarizing, clearing and editing earlier content all change the prefix, so each one costs a cache write. Appending new content leaves the prefix untouched. The rest of this lesson weighs each technique on both questions. For how cached reads are priced, see cost architecture.
Checkpoint 1 of 9· Check yourself
An incident-triage agent has a fully cached 150k-token prefix and is nearing its 1M-token context window as stale log output piles up. Which statement about the team's options is correct?
Caching changes what the tokens cost, not whether they take up space in the window. Clearing earlier content shrinks the window, but it also changes the cached prefix, so you pay a cache write each time content is cleared.
“You’ll incur cache write costs each time content is cleared, but subsequent requests can reuse the newly cached prefix.”Source: platform.claude.com
2.Choosing a compaction mode
Compaction replaces the older turns of a conversation with a summary that Claude writes on the server, so you don't write summarization code yourself. There are three ways to run it, and they differ mainly in who decides when compaction happens and whether the user waits for it.
| Aspect | Compaction on demand | Compaction at a token threshold | Your own summarizer |
|---|---|---|---|
| Who decides when | You, by sending a request | The API, when input tokens reach the trigger you set | You |
| Runs in the background | Yes | No: it runs inside the request that reaches the threshold | Yes, in your own code |
| Recent turns stay word for word | Yes, with compaction that keeps recent turns | Yes, by pausing after compaction and reinserting them | Yes |
| Choose it when | You must control timing, can’t pause while a summary is written, or must keep recent turns and their thinking | You want the API to manage context inside ordinary requests | You already run your own summarizer |
Lowering the trigger only moves the stall. Threshold compaction runs inside the request that reaches the threshold. Wherever the threshold sits, one user request still carries the summarization, so the user waits whenever it fires. Background compaction is the fix. You send an on-demand compaction request with the history as it stands and record how many messages it held, and the conversation keeps running on its full history. When the response arrives with stop_reason "compaction", you drop exactly the messages you sent from the front of the history and put the returned block in their place. Turns added in the meantime stay after it. The conversation keeps growing until the swap, so start the request while the window still has room for those turns. The compaction request also counts against your rate limits like any other request.
Two refinements fix the most common complaint about summaries, which is lost detail. If a debugging agent forgets the exact error text and the half-finished edit from its last three turns, use compaction that keeps recent turns. The summary then covers only the older turns, and the last few reach Claude word for word. If the default summary keeps dropping one kind of fact that later turns need, write your own summarization prompt, which replaces the default one.
The TypeScript and Ruby SDKs also offer a client-side compaction through tool_runner with compaction_control. It replaces the entire history with a summary once total tokens pass context_token_threshold (default 100,000). Server-side compaction is generally preferred. The client-side version suits long tasks that make measurable progress and produce artifacts outside the conversation. It is a poor fit for tasks that need precise recall of early details or exact state across many variables. It also misfires with server-side tools: its total includes cache_read_input_tokens accumulated across internal calls, so a 63,000-token context can be counted as 334,400 tokens and compacted too early.
Checkpoint 2 of 9· Match them up
A team is choosing how to keep a long-running agent's history small. Match each summarization approach to how it behaves.
Tap a term, then the definition that fits it.
Threshold compaction runs inside the request that crosses the trigger, while on-demand compaction lets you choose the timing and run it in the background. Your own summarizer runs in your own code. The SDK's compaction_control replaces the entire history, and because its total includes accumulated cache_read_input_tokens, it can compact too early when server-side tools are in use.
“No: it runs inside the request that reaches the threshold”Source: platform.claude.com
Checkpoint 3 of 9· Exam question
Fennel Health runs a production patient-support chat assistant that helps members with benefits questions, claim follow-ups, and appointment changes. Sessions are dialogue-heavy, with few tool calls, and often run for hours. The product summarizes the conversation once a token threshold is crossed. When a long session hits that trigger, the user waits many seconds while the summary is written inside that same request, and support tickets describe the chat as frozen. The team already moved the threshold earlier, but members still see the same stall, only more often. Members also rely on details stated early, such as consent preferences and agreed callback times, so those must survive in later turns. Product requires that the chat never visibly pause mid-conversation. What is the best change?
Correct answer: A — Run on-demand compaction in the background before the limit and swap the summary in when it returns, so the session never pauses.
- A. Correct. Background compaction runs while work continues and is swapped in on arrival, so the user-facing request does not carry the summarization cost.
- B. Earlier triggering still runs the summary inside an ordinary request, so the user still waits, just more often and on a slightly shorter history.
- C. Dropping the oldest turns is instant but discards context outright, including decisions the session may still depend on, and can split tool call pairs.
- D. Clearing removes only tool results. A dialogue-heavy chat would barely shrink and would still hit the window limit.
3.Clearing stale tool results and thinking blocks
Context editing removes specific content instead of summarizing. It suits agents whose bulk is stale tool output, such as file reads and search results that Claude has already processed. The clear_tool_uses_20250919 strategy clears the oldest tool results in chronological order once the prompt passes a trigger, and replaces each one with placeholder text. By default only the results are cleared. Claude's original tool calls stay visible, and assistant turns holding design decisions are untouched. Edits are applied server-side before the prompt reaches Claude. Your client keeps the full, unmodified history and does not need to sync with the edited version.
| Parameter | Default | What it controls |
|---|---|---|
| trigger | 100,000 input tokens | When clearing starts; set in input_tokens or tool_uses |
| keep | 3 tool uses | How many recent tool use/result pairs survive |
| clear_at_least | None | Minimum tokens per clear; if not reachable, the strategy isn’t applied |
| exclude_tools | None | Tools whose uses and results are never cleared |
| clear_tool_inputs | false | Whether tool call parameters are cleared along with results |
context_management={
"edits": [
{
"type": "clear_tool_uses_20250919",
# Trigger clearing when threshold is exceeded
"trigger": {"type": "input_tokens", "value": 30000},
# Number of tool uses to keep after clearing
"keep": {"type": "tool_uses", "value": 3},
# Optional: Clear at least this many tokens
"clear_at_least": {"type": "input_tokens", "value": 5000},
# Exclude these tools from being cleared
"exclude_tools": ["web_search"],
}
]
},Clearing has a cost: each clear invalidates the cached prompt prefix, and you pay a cache write to rebuild it. Consider a triage agent with a low trigger that frees about 2k tokens per clear. It breaks the cache almost every turn and saves very little each time, so cache reads fall and the bill goes up. The fix is clear_at_least set to a large token count. Clears then happen rarely and in large batches, and an edit that can't free that much is skipped. The other options don't fix it. Lowering the trigger makes clears more frequent. Raising keep changes how many results survive, not the minimum batch size. clear_tool_inputs frees a little more per clear but still lets small, frequent clears through. The token counting endpoint accepts the same context_management settings and reports original_input_tokens alongside the post-clearing count, so you can measure the saving before you commit to it.
Thinking-block clearing makes the same trade between cache and window. Thinking that stays in context keeps the cache intact. Clearing it invalidates the cache from the point where it was cleared. A team that adds the strategy to save tokens can watch prompts shrink while cache reads collapse after every turn. If cache performance matters more than window space, set keep to "all". Otherwise set a larger {type: "thinking_turns", value: N}. Both strategies can run in the same edits list. On models with preserved thinking, server-side context management never invalidates thinking blocks. Client-side edits to earlier turns can invalidate the thinking in every later assistant turn.
Checkpoint 4 of 9· Check yourself
A coding agent turns on clear_tool_uses_20250919 with a 20k-token trigger. Each clear frees about 2k tokens. Prompts get smaller, but cache reads fall sharply and the bill goes up. Which configuration change fixes this?
Every clear invalidates the cached prefix and costs a cache write, so small, frequent clears cost more than they save. clear_at_least sets a minimum size for each clear and skips any edit that can't reach it. Lowering the trigger makes clears more frequent. keep and clear_tool_inputs don't set a minimum batch size, so small clears still get through.
“Use the clear_at_least parameter to ensure a minimum number of tokens is cleared each time.”Source: platform.claude.com
Checkpoint 5 of 9· Exam question
Orchid Software runs an incident-triage agent in production that reads large log files while diagnosing outages. By turn 25 the prompt exceeds 100k tokens, mostly stale log reads that the agent never consults again after a few turns. The team enabled tool-result clearing with a low trigger. Billing now shows cache read tokens falling and cost rising, and traces show a clear firing on nearly every turn, each freeing only about 2k tokens. A trial that kept far more recent results in context left the prompt above 100k tokens and did not reduce cost. Engineers want the prompt to shrink substantially while the cached prefix is rewritten far less often. What is the best change?
Correct answer: C — Set the clear_at_least parameter to a high token count, so each edit removes a large batch of old results and cache rewrites happen less often.
- A. A larger keep value does reduce how often clearing fires, but it leaves nearly all of the stale log reads in the prompt. The context stays above 100k tokens and every turn still pays for it.
- B. A lower trigger makes clearing fire sooner and more often. Each edit still frees only about 2k tokens but changes the prompt prefix, so more cache writes happen and cache reads fall further.
- C. Correct. Every clear invalidates the cached prefix. A high minimum-to-clear value makes the strategy skip edits that would save too little, so clearing happens in fewer, larger batches that justify the cache rewrite.
- D. Clearing tool inputs frees somewhat more per edit, but it does not control when edits fire, so small clears still break the cache as often. It also removes the call parameters Claude uses to recall what it already tried.
Sources2
4.Persistent memory outside the window
Compaction and clearing both throw information away. The memory tool keeps it somewhere else. With {"type": "memory_20250818", "name": "memory"} enabled, Claude can create, read, update and delete files under /memories that persist between conversations. The tool is client-side: Claude only requests file operations, and your application runs them against storage you control. /memories is a prefix your handler maps onto real storage, such as a per-user directory or keys in a database. A later conversation picks up the same memory only when it sends the same tools entry and your handler serves the same store. Restrict every operation to the /memories directory to block path traversal.
Memory supports just-in-time retrieval instead of preloading. Claude checks its memory directory before starting a task, records what it learns as it goes, and reads files back only when it needs them. Memory works well alongside tool-result clearing. As context approaches the clearing threshold, Claude gets an automatic warning to preserve important information. It can then write the essentials of a large file read to memory before the result is replaced with a placeholder, and look them up again later. Resuming the conversation itself is a different mechanism, covered under durable sessions.
Checkpoint 6 of 9· Check yourself
After the memory tool is turned on, Claude saves notes under /memories during a conversation. The next day, a new conversation starts with an empty memory directory. The team's handler maps /memories to a temporary directory that is created fresh for each conversation. What explains the missing notes?
The memory tool runs on the client side. Claude asks for file operations, and the application performs them against storage it controls. If the handler points each conversation at a new temporary directory, a later conversation has nothing to read. Claude checks its memory before a task starts, not only when the clearing warning appears.
“A later conversation continues from the same memory when it sends the same tools entry and your handler serves the same store.”Source: platform.claude.com
5.Changing instructions and tools without breaking the cache
The prompt cache hashes the request in a fixed order: tools first, then the top-level system field, then the messages. Editing anything early in that order invalidates the cache for everything after it. Editing system invalidates every cached turn. Editing the tools array, which sits even earlier, invalidates the whole conversation. Suppose a refund policy tightens 80 turns into a session. Rewriting the system prompt sends cache reads to zero on every request.
Append a {"role": "system"} message at the point where the instruction becomes relevant. Tools, system and earlier turns stay byte-identical, so the next request still reads the prefix from cache. The new text still applies as a system instruction rather than as ordinary user text. The alternatives all fail. Adding a cache breakpoint after edited system text still leaves a changed prefix. Prepending the policy to the first user message edits an early turn, which invalidates everything after it. Starting a fresh conversation with a summary pays a full cache write and loses the verbatim history.
Tool availability follows the same pattern. Declare the full tool set in tools up front, and never change that array. Then put tool_addition and tool_removal blocks in the content of a system-role message to offer or withdraw a tool from that point on. Each block references a tool by name (tool_reference), or references MCP tools individually or as a whole toolset. A name that isn't declared in tools returns a 400 error. If you declare a known tool with defer_loading: true, it stays withheld until a tool_addition surfaces it. Under the inline-tools-2026-09-15 beta header, a tool_addition can instead carry a full definition. That lets you introduce a tool you didn't know about at the start, or change a tool's schema. Sending the identical definition again changes nothing, so retries are safe.
{
"role": "system",
"content": [
{
"type": "tool_addition",
"tool": {
"type": "tool_definition",
"definition": {
"name": "db_query",
"description": "Run a read-only SQL query against the analytics database.",
"input_schema": {
"type": "object",
"properties": { "sql": { "type": "string" } },
"required": ["sql"]
}
}
}
}
]
}Define tools by value only when you have to. A tool you know about at the first request belongs in tools, deferred if you don't want to offer it yet. Keep at least one non-deferred tool in tools. If every tool is deferred, the request fails with a 400, unless it sends the inline-tools header and the messages include a tool_addition. Even then, the first tool defined by value costs a full cache miss.
Checkpoint 7 of 9· Fill the gap
Eighty turns into a cached session, your application must give the model a new db_query tool without invalidating any cached turn. Which role completes the message you append?
{
"role": " ? ",
"content": [
{
"type": "tool_addition",
"tool": {
"type": "tool_definition",
"definition": {
"name": "db_query",
"description": "Run a read-only SQL query against the analytics database.",
"input_schema": {
"type": "object",
"properties": { "sql": { "type": "string" } },
"required": ["sql"]
}
}
}
}
]
}tool_addition and tool_removal blocks go in the content of an appended system-role message. The tools array, the top-level system field and earlier turns stay byte-identical, so the cached prefix still hits. These blocks don't belong in a user or assistant turn, and putting a tool change in user text would not apply it as a system-level change.
Source: platform.claude.comCheckpoint 8 of 9· Exam question
Ember Games runs a player-support agent in production that issues refunds through tool calls. One long-running session is now 80 turns in and carries a large cached prefix of tool definitions, the system prompt, and verbatim ticket history that auditors may review. Finance lowers the refund amount the agent may approve alone: any refund above the new limit must be escalated to a human, starting with the very next request. In staging, the team tested a revision of the top-level system prompt, and every such edit sent cache reads to zero, with the full prefix billed as a fresh write on the following turn. The model supports whichever mechanism the team picks, and the session's verbatim history must stay intact. On a model that supports it, what is the cache-safe approach?
Correct answer: C — Append a system-role message carrying the new policy to the end of the messages, leaving tools, system and earlier turns byte-identical.
- A. A system edit invalidates the system level and every message after it, so a new breakpoint does not save the cached conversation. It is rewritten in full.
- B. Changing an early message alters the prefix from that point onward, so the whole cached conversation after it is invalidated and rewritten.
- C. Correct. Appending a system-role message leaves everything before it unchanged, so the cached prefix still hits while the new instruction applies from that point.
- D. A fresh conversation loses verbatim history and pays a full cache write on a summarized prefix. The rule can be changed in place without it.
Sources6
6.Keeping a large tool catalog out of the window
Tool definitions take up context too. A multiserver setup with GitHub, Slack, Sentry, Grafana and Splunk can use about 55k tokens of definitions before Claude does any work. Accuracy suffers as well: Claude's ability to pick the right tool degrades once more than 30 to 50 tools are available. The tool search tool addresses both problems. You add tool_search_tool_regex_20251119, where Claude writes regex patterns, or tool_search_tool_bm25_20251119, where Claude writes natural-language queries. Then you mark the rest of the catalog defer_loading: true. Claude starts with only the search tool and the non-deferred tools. It searches names, descriptions, argument names and argument descriptions, and receives matches as tool_reference blocks (up to 5 by default), which the API expands into full definitions. Tool search typically cuts definition tokens by over 85 percent.
{
"name": "get_weather",
"description": "Get current weather for a location",
"input_schema": {
"type": "object",
"properties": {
"location": { "type": "string" },
"unit": { "type": "string", "enum": ["celsius", "fahrenheit"] }
},
"required": ["location"]
},
"defer_loading": true
}defer_loading decides what enters the context window, not what goes in the request. You still send every tool's full definition on every request, because the API needs them server-side to run the search and expand references. Never defer the tool search tool itself. Keep your 3 to 5 most-used tools non-deferred so Claude can call them without searching first. The design also protects the cache. The API excludes deferred tools from the system-prompt prefix, and a discovered tool is appended inline in the conversation as a tool_reference. Discovering a tool therefore adds to the end of the prompt and leaves the cached prefix alone.
Checkpoint 9 of 9· Fill the gap
Your agent sends 200 tool definitions. You want get_weather to stay out of the context window until Claude finds it through tool search, but its full definition still goes in the tools array on every request. Which field completes the definition?
{
"name": "get_weather",
"description": "Get current weather for a location",
"input_schema": {
"type": "object",
"properties": {
"location": { "type": "string" },
"unit": { "type": "string", "enum": ["celsius", "fahrenheit"] }
},
"required": ["location"]
},
" ? ": true
}defer_loading: true keeps a tool out of the context window until Claude finds it through search. The full definition is still sent on every request, because the API needs it on the server to run the search and expand tool_reference blocks. cache_control sets cache breakpoints and strict constrains tool-call output. Neither one withholds a tool from the window.
Source: platform.claude.comSources7
Sources
(7)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“it keeps the active context small, because response quality degrades as a conversation grows”
↩︎ Why a long-running agent needs a context policy“Compaction replaces the older turns of a conversation with a summary that Claude writes on the server”
↩︎ Choosing a compaction mode“No: it runs inside the request that reaches the threshold”
↩︎ Choosing a compaction mode“Compaction that keeps recent turns: when the last few turns must reach Claude word for word. The summary covers only the older turns.”
↩︎ Choosing a compaction mode - 2.
“context is a finite resource with diminishing returns, and irrelevant content degrades model focus”
↩︎ Why a long-running agent needs a context policy“the SDK might incorrectly calculate token usage, causing compaction to trigger at the wrong time.”
↩︎ Choosing a compaction mode“When activated, the API automatically clears the oldest tool results in chronological order.”
↩︎ Clearing stale tool results and thinking blocks“Your client application maintains the full, unmodified conversation history.”
↩︎ Clearing stale tool results and thinking blocks“If the API can’t clear at least the specified amount, the strategy isn’t applied.”
↩︎ Clearing stale tool results and thinking blocks“When thinking blocks are cleared, the cache is invalidated at the point where clearing occurs.”
↩︎ Clearing stale tool results and thinking blocks“Claude receives an automatic warning to preserve important information.”
↩︎ Persistent memory outside the window“on every model, including models whose per-model default keeps all thinking”
↩︎ Exam trap 2“Invalidates cached prompt prefixes when content is cleared. To account for this, clear enough tokens to make the cache invalidation worthwhile.”
↩︎ Exam trap 3“You’ll incur cache write costs each time content is cleared, but subsequent requests can reuse the newly cached prefix.”
↩︎ Checkpoint“on every model, including models whose per-model default keeps all thinking”
↩︎ Prediction“Use the clear_at_least parameter to ensure a minimum number of tokens is cleared each time.”
↩︎ Checkpoint - 3.
“Cached prompt prefixes still occupy the context window: prompt caching changes what you pay for those tokens, not whether they count.”
↩︎ Why a long-running agent needs a context policy“Cached prompt prefixes still occupy the context window: prompt caching changes what you pay for those tokens, not whether they count.”
↩︎ Key concept“Cached prompt prefixes still occupy the context window: prompt caching changes what you pay for those tokens, not whether they count.”
↩︎ Exam trap 1 - 4.
“the compaction request runs while the conversation continues on its full history, and the swap waits until the block arrives”
↩︎ Choosing a compaction mode“start the compaction request while the context window still has room for the turns that arrive meanwhile”
↩︎ Choosing a compaction mode - 5.
“The memory tool operates client-side: Claude requests file operations, and your application runs them.”
↩︎ Persistent memory outside the window“The /memories path is a prefix that your handler maps onto real storage, such as a per-user directory or keys in a database.”
↩︎ Persistent memory outside the window“an agent records what it learns in memory files and reads them back on demand”
↩︎ Persistent memory outside the window“A later conversation continues from the same memory when it sends the same tools entry and your handler serves the same store.”
↩︎ Checkpoint - 6.
“The tools array sits even earlier in the hashed request prefix than the top-level system field”
↩︎ Changing instructions and tools without breaking the cache“the new instruction is still applied as a system instruction rather than as ordinary user text”
↩︎ Changing instructions and tools without breaking the cache“The tools array itself never changes, so the cached prefix stays intact.”
↩︎ Changing instructions and tools without breaking the cache“Sending an identical definition again changes nothing, so a client can safely resend it, for example on a retry.”
↩︎ Changing instructions and tools without breaking the cache“editing the top-level system field changes the very beginning of the prompt and invalidates the cache for everything that follows”
↩︎ Exam trap 4 - 7.
“Claude’s ability to pick the right tool degrades once you exceed 30–50 available tools.”
↩︎ Keeping a large tool catalog out of the window“defer_loading controls what enters the context window, not what you send in the request”
↩︎ Keeping a large tool catalog out of the window“Keep your 3–5 most frequently used tools non-deferred so Claude can call them without searching first.”
↩︎ Keeping a large tool catalog out of the window“Internally, the API excludes deferred tools from the system-prompt prefix.”
↩︎ Keeping a large tool catalog out of the window“defer_loading controls what enters the context window, not what you send in the request”
↩︎ Exam trap 5