What you will be able to do
- Explain why both too much and too little recalled context lower agent answer quality
- Recognise the symptoms and memory-system causes of over-retrieval (context pollution)
- Recognise the symptoms and memory-system causes of under-retrieval (missing relevant history)
Key concept
Context composition risk — An agent's answer quality depends on what its memory retrieval puts into the context window. Retrieval can fail in two opposite directions. Over-retrieval adds superfluous or wrong material that pollutes the context. Under-retrieval leaves out history the agent needed.
1.Memory retrieval can fail in two directions
A memory system decides what past information goes back into the model's context. On Databricks there are two Lakebase-backed stores for this. Managed agent sessions hold the state of one interaction, usually the ordered transcript of messages, tool calls and results. Managed agent memory holds durable facts, preferences and decisions that the agent recalls in later, unrelated conversations. The scoping differs: sessions are scoped to one interaction, and memories are scoped to a subject, not an interaction. Within a memory store, a search for an actor returns the most relevant entries ranked by a full-text (BM25) relevance score.
That is the core of this objective. Over-retrieval (context pollution) puts material into the context that does not help the current request and can mislead the model. Under-retrieval (missing relevant history) leaves out a fact, preference or earlier turn that the answer depended on. You cannot fix one by maximising or minimising retrieval, because each fix moves you toward the other failure. The job is to diagnose which failure you have.
Checkpoint 1 of 4· Check yourself
A teammate says the way to protect answer quality is to tune retrieval toward the largest possible recall, since only missing history hurts. Which response matches the guidance?
Both missing important information and superfluous information make high-quality output difficult, so retrieval has to be diagnosed in both directions rather than simply maximised.
“the context provided to the LLM is missing important information or contains superfluous information”Source: docs.databricks.com
2.Over-retrieval: when recalled context pollutes the answer
Databricks' write-up on agent harnesses lists two over-retrieval failures among the most common operational problems. The first is context rot: as conversation history grows, the model's reasoning quality degrades, and without a strategy to trim or summarize older context, performance often breaks down on long-running tasks. The second is irrelevant retrieval: when the harness pulls in the wrong information from memory or search systems, the model may confidently generate incorrect answers. The word *confidently* matters. Polluted context does not make the agent hesitate. It produces wrong answers that look right.
In a Databricks memory design, over-retrieval usually comes from one of three places:
- The session is treated as long-term memory. A session grows within the interaction. Replaying an ever-growing transcript is the context-rot pattern. The intended design is different: as a session accumulates, the agent distills the durable facts out of it and writes them to memory.
- Too many entries are returned per search. The number of documents retrieved is one of the tuning knobs on the chain side, and everything you retrieve must fit within the LLM context window.
- The search is scoped to the wrong set of entries. Every list and search is scoped to a single actor_id. If a request without a verified user identity falls back to a shared actor_id, the agent recalls other people's memories. The guidance is to reject such requests rather than fall back.
Checkpoint 2 of 4· Check yourself
An agent's answers get worse the longer a single conversation runs, even though nothing changed in its memory store. Which diagnosis fits best?
Quality that degrades as conversation history grows is context rot, a form of over-retrieval. The fix is to trim or summarize older context, not to retrieve more.
“Without a strategy to trim or summarize older context, performance often breaks down on long-running tasks.”Source: www.databricks.com
3.Under-retrieval: when relevant history never reaches the model
The opposite failure is quieter. For RAG agents, missing key context leads to incorrect answers or hallucinations. The model fills the gap instead of saying it does not know. In a memory system, relevant history goes missing in a few recognisable ways:
- It was never written to memory. Sessions are cleared or age out, and only distilled memory persists. If the agent never saved a durable preference, it cannot recall it next week.
- It was written but the search does not find it. Search ranks entries by full-text (BM25) relevance. A query phrased in different words from the stored entry can miss it. That is why the entry's description field matters: the docs say it improves retrieval.
- The search is scoped too narrowly. A wrong actor_id, or a path prefix that excludes the right folder when listing, returns nothing useful even though the entry exists.
- Too few entries are returned. The same retrieval-count knob that causes over-retrieval when it is set too high causes under-retrieval when it is set too low.
Query understanding helps with the wording problem. Query rewriting can paraphrase conversation history in a multi-turn chat, or replace words in the user query with synonyms to capture a broader range of relevant documents. Rewriting widens recall, so it also needs watching for the over-retrieval it can bring back in.
| Failure mode | What the model sees | Typical memory-system cause | Typical symptom |
|---|---|---|---|
| Over-retrieval (context pollution) | Superfluous or wrong entries alongside the useful ones | Full transcript replayed every turn; search returns too many entries; fallback to a shared actor_id | Confident but wrong answers; quality falls as conversations grow; rising token counts |
| Under-retrieval (missing history) | The needed fact or earlier decision is absent | Fact never distilled from the session; query wording misses BM25 match; weak description; scope or limit too narrow | Incorrect answers or hallucinations; agent ignores a known preference |
Checkpoint 3 of 4· Match them up
Match each observation to the failure it most likely indicates
Tap a term, then the definition that fits it.
Missing distillation leaves nothing to recall. A wrong or shared actor_id pulls in another user's memories. An untrimmed transcript causes context rot.
“As a session accumulates, an agent (or your application) distills the durable facts out of it and writes them to memory.”Source: docs.databricks.com
Checkpoint 4 of 4· Exam question
A team sets an aggressive similarity-score cutoff on their memory retriever so that only near-exact matches are returned. Afterward, users notice the agent frequently fails to recall clearly relevant past preferences that were phrased differently than the current query. What is happening, and what should the team do?
Correct answer: A — The strict threshold is causing under-retrieval by excluding relevant but loosely worded memories; relax the threshold or broaden the matching strategy
- A. Correct: an overly strict similarity cutoff filters out relevant memories that are phrased differently but semantically related, producing under-retrieval. Relaxing the threshold or using a broader matching approach (e.g., hybrid search) restores recall.
- B. Incorrect: the symptom described is missing relevant memories, not noise from excess irrelevant matches, so this is not over-retrieval, and tightening the threshold further would only exclude more relevant results.
- C. Incorrect: nothing in the scenario points to storage capacity as the cause; the described failure is a matching/threshold issue, not a capacity issue.
- D. Incorrect: while embedding quality can affect matching, the scenario specifically describes a threshold that excludes valid matches, which is a configuration issue independent of embedding model version.
- E. Incorrect: a full context window would cause truncation of retrieved or instructional content generally, not a pattern where semantically relevant-but-differently-phrased memories are specifically excluded by score.
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.Retrieving more memory is the safe default, because the model can ignore what it does not need.Why is that wrong?
Superfluous context is a quality problem in its own right, just as missing context is. Extra irrelevant material can lead the model to wrong answers.
Covered in Memory retrieval can fail in two directions
2.Keeping the session history is enough for the agent to remember a user's preferences in future conversations.Why is that wrong?
Sessions are scoped to one interaction and are later cleared or age out. Only facts distilled into the memory store carry over to separate conversations.
Covered in Under-retrieval: when relevant history never reaches the model
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“Memories are scoped to a subject, not an interaction”
↩︎ Memory retrieval can fail in two directions“The session is later cleared or ages out; the memory persists.”
↩︎ Exam trap 2“As a session accumulates, an agent (or your application) distills the durable facts out of it and writes them to memory.”
↩︎ Checkpoint - 2.
“Search returns the most relevant entries ranked by a full-text (BM25) relevance score.”
↩︎ Memory retrieval can fail in two directions“If your strategy depends on an end-user identity, reject requests that don't carry one rather than falling back to a shared actor_id.”
↩︎ Over-retrieval: when recalled context pollutes the answer“and description improves retrieval”
↩︎ Under-retrieval: when relevant history never reaches the model - 3.https://www.databricks.com/blog/ai-harnessSecondary source
“When the harness pulls in the wrong information from memory or search systems, the model may confidently generate incorrect answers.”
↩︎ Over-retrieval: when recalled context pollutes the answer“Without a strategy to trim or summarize older context, performance often breaks down on long-running tasks.”
↩︎ Checkpoint - 4.
“Context limits: Must fit within LLM context window when retrieving multiple chunks.”
↩︎ Over-retrieval: when recalled context pollutes the answer“RAG agents: Missing key context leads to incorrect answers or hallucinations.”
↩︎ Under-retrieval: when relevant history never reaches the model - 5.
“The retrieval parameters (for example, the number of chunks or documents retrieved).”
↩︎ Over-retrieval: when recalled context pollutes the answer“the context provided to the LLM is missing important information or contains superfluous information”
↩︎ Key concept“the context provided to the LLM is missing important information or contains superfluous information”
↩︎ Exam trap 1“the context provided to the LLM is missing important information or contains superfluous information”
↩︎ Prediction - 6.
“Replacing words or phrases in the user query with synonyms to capture a broader range of relevant documents”
↩︎ Under-retrieval: when relevant history never reaches the model