What you will be able to do
- Choose between trimming, summarizing, clearing or forking session history to shrink a context that has grown too large
- Move durable facts out of a session into managed memory so the session can be cleared without losing them
- Apply retrieval-side and fresh-start fixes, and confirm the fix restored quality
1.Compaction: trim or summarize what the model sees
If an agent fails because its context keeps growing, the direct fix is to stop it growing unchecked. Databricks' harness article calls this context compaction: as conversations grow longer, the harness decides what stays active and what gets summarized. In practice, that means trimming older parts of the conversation so the model is not overwhelmed. The model does not change. What changes is the slice of history it receives on each turn.
Managed agent sessions provide low-level operations over a conversation's history. A session store keeps a session's items in a fixed order. Frameworks read the history at the start of a run and append new items at the end. The clients also let you remove the most recent item, clear a session's items, or fork a conversation into an independent copy, optionally up to a specific item. The documentation lists these operations but does not prescribe which one to use for compaction. Choosing among them, and deciding what to summarize, is a design decision for your harness.
| Framework operation | Session store call |
|---|---|
| Read history | list_items in chronological order (order_by="create_time asc") |
| Add turn items | append the new items |
| Undo last item | pop the most recent item |
| Clear the thread | clear the session's items |
Selecting the intervention starts with the symptom you diagnosed. The table below maps each symptom to the fix the sources support. Several can apply at once, so confirm each one with its own measurement.
| Diagnosed symptom | Intervention | Measure to confirm |
|---|---|---|
| Retrieval gets worse in later turns because follow-ups depend on earlier conversation | Rewrite the query from conversation history, together with changes to the retrieval component | chunk_relevance/precision and document_recall |
| Reasoning quality degrades as conversation history grows (context rot) | Compaction: trim or summarize older context | Session-level judges and token counts at the same turn positions |
| Facts from early turns are lost or contradicted later | Distill durable facts into managed memory before trimming or clearing the session | KnowledgeRetention |
| Standing context (columns, instructions, examples) is near the token limit | Cut those tokens, for example remove unneeded columns and prune redundant examples | Whether the token-limit warning clears and answer quality recovers |
Checkpoint 1 of 4· Check yourself
Which managed-sessions capability creates an independent copy of a conversation, optionally up to a specific item?
A fork produces an independent copy and can stop at a chosen item. Popping removes the latest item and clearing removes the session's items.
“The clients also support removing the most recent item, clearing a session's items, and forking a conversation into an independent copy”Source: docs.databricks.com
2.Distill durable facts into memory, then let the session go
Trimming carries a risk: a fact the user gave in turn three may be needed in turn fifty. Databricks handles this with two stores. Sessions hold the working state of one interaction. Managed agent memory holds durable facts, preferences and decisions, which are retrieved by semantic search in later conversations. The two connect when something in a conversation is worth keeping. As a session accumulates, the agent or application distills the durable facts and writes them to memory. The session is later cleared or ages out, and the memory persists.
For a long interaction, this means you do not have to choose between a bloated context and a forgetful agent. Facts that matter beyond the current task go into memory and are recalled when relevant. The session can then be trimmed or cleared, so each turn reads a smaller context. The stores are independent: deleting a session, or a whole session store, does not delete memory retained in a memory store.
Checkpoint 2 of 4· Check yourself
After writing a user's stable preferences to managed memory, an engineer clears the long session to shrink the context. What happens to those preferences?
Memory is scoped to a subject rather than an interaction, and deleting a session does not touch the memory store.
“deleting a session, or an entire session store, does not delete memory retained in a memory store.”Source: docs.databricks.com
Sources4
3.Retrieval-side fixes and a fresh start
If the damage shows up in retrieval accuracy, you may not need to cut history at all. The query just needs to stop depending on it. Databricks' RAG quality guide lists "paraphrasing conversation history in a multi-turn chat" as an example of query rewriting: turning the user's question into one that better matches the intent and the retrieval documents. The guide also notes that splitting query understanding across several LLM calls, for example classifying intent, then extracting entities, then rewriting, can give better results than one large prompt. The cost is some extra latency.
Sometimes the simplest fix is a clean slate. Genie's troubleshooting guide says previous interactions can influence responses within a chat, and that starting a new chat gives you a blank starting point. Separately, in its advice on exceptionally long queries and timeouts, it lists starting a new chat if Genie generates slow or failing responses. Keep this distinct from token reduction. The guide's token-count advice (remove unnecessary columns, streamline column descriptions, prune redundant example SQL queries, simplify instructions) targets standing context, meaning the instructions and metadata configured in the agent. It does not address conversation length. Diagnose which of the two is growing before choosing the fix.
Checkpoint 3 of 4· Check yourself
A Genie Agent starts to generate slow or failing responses. Which action does the Genie troubleshooting guide list for this?
The guide lists starting a new chat for slow or failing responses. It is not specific to long conversations, so confirm the cause before relying on it. Adding examples or redundant descriptions raises the token count instead of lowering it.
“Start a new chat if Genie starts to generate slow or failing responses.”Source: docs.databricks.com
4.Confirm the intervention restored performance
An intervention is only finished when you have measured its effect. Databricks frames agent quality work as a repeating loop, and context-length fixes follow it. Find the turn where quality drops. Measure it with session-grouped traces. Apply compaction, distillation to memory, query rewriting or a fresh start. Then re-run the same session-level judges and retrieval metrics on the new version, and keep monitoring so the problem does not come back as conversations grow again.
To make the before and after comparable, run the same scenarios against both versions, either pre-recorded conversations from the previous version or simulated conversations with consistent goals. Compare scores at the same turn positions, since a fix that only helps early turns has not addressed late-session degradation. As a practical suggestion, also track token counts per turn: a drop in tokens with a drop in KnowledgeRetention means the trimming removed something that still mattered.
Checkpoint 4 of 4· Check yourself
You changed how a long-running agent compacts its history. Which evaluation approach best shows whether the change restored quality?
Consistent scenarios let you attribute any difference to the change. Token counts alone cannot show whether quality recovered, and first-turn scores miss late-session degradation.
“Test a new agent version systematically with consistent scenarios”Source: docs.databricks.com
Model answer for the reflection: the trimming dropped facts that still mattered, so distill durable facts from the session into managed memory before trimming, so they can be recalled when relevant. Then re-run the KnowledgeRetention judge on the same scenarios and compare it at the same turn positions, while checking that token counts stay low.
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.Clearing or deleting a long session to shrink the context also wipes whatever the agent saved to memory.Why is that wrong?
Sessions and memory are independent stores. Durable facts distilled into memory outlive the session.
Covered in Distill durable facts into memory, then let the session go
2.Adding a step that rewrites queries from conversation history is a complete fix for poor retrieval in long chats.Why is that wrong?
Databricks requires query rewriting to be paired with changes to the retrieval component.
Covered in Retrieval-side fixes and a fresh start
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.https://www.databricks.com/blog/ai-harnessSecondary source
“As conversations grow longer, the harness decides what stays active and what gets summarized, a process known as context compaction.”
↩︎ Compaction: trim or summarize what the model sees“trimming older parts of the conversation so the model does not become overwhelmed as the context grows”
↩︎ Compaction: trim or summarize what the model sees - 2.
“The clients also support removing the most recent item, clearing a session's items, and forking a conversation into an independent copy”
↩︎ Compaction: trim or summarize what the model sees - 3.
“Does the agent correctly retain information from earlier in the conversation?”
↩︎ Compaction: trim or summarize what the model sees - 4.
“As a session accumulates, an agent (or your application) distills the durable facts out of it and writes them to memory.”
↩︎ Distill durable facts into memory, then let the session go“The session is later cleared or ages out; the memory persists.”
↩︎ Exam trap 1“deleting a session, or an entire session store, does not delete memory retained in a memory store.”
↩︎ Checkpoint - 5.
“Paraphrasing conversation history in a multi-turn chat”
↩︎ Retrieval-side fixes and a fresh start“breaking down the query understanding process into multiple LLM calls can lead to better results”
↩︎ Retrieval-side fixes and a fresh start“Query rewriting must be done in conjunction with changes to the retrieval component”
↩︎ Exam trap 2“Query rewriting must be done in conjunction with changes to the retrieval component”
↩︎ Prediction - 6.
“Previous interactions might influence Genie's responses in any given chat, but starting a new chat gives you a blank starting point”
↩︎ Retrieval-side fixes and a fresh start“Remove unnecessary columns: Unnecessary columns in your tables can significantly contribute to token usage.”
↩︎ Retrieval-side fixes and a fresh start“Start a new chat if Genie starts to generate slow or failing responses.”
↩︎ Checkpoint - 7.
“find an issue in your agent, measure it systematically, fix it, confirm the fix, and monitor so it doesn't recur.”
↩︎ Confirm the intervention restored performance - 8.
“Test a new agent version systematically with consistent scenarios”
↩︎ Confirm the intervention restored performance