What you will be able to do
- Implement Contextual Embeddings and Contextual BM25, and keep their cost low with prompt caching
- Add a reranking stage and tune the trade-off between accuracy, latency and cost
- Pass retrieved chunks to Claude as search_result blocks so its answers carry citations
1.Contextual retrieval: add context before you index
Chunks lose the context of the document they came from, and that hurts retrieval. Contextual Retrieval addresses this at indexing time. For each chunk, Claude receives the whole document plus the chunk and writes a short, chunk-specific context, usually 50–100 tokens. That text is prepended to the chunk before it is embedded (Contextual Embeddings) and before it goes into the BM25 index (Contextual BM25). A filing chunk that said only 'revenue grew by 3%' now opens by naming ACME Corp, Q2 2023 and the previous quarter's revenue. This is the prompt Anthropic used:
<document>
{{WHOLE_DOCUMENT}}
</document>
Here is the chunk we want to situate within the whole document
<chunk>
{{CHUNK_CONTENT}}
</chunk>
Please give a short succinct context to situate this chunk within the overall document for the purposes of improving search retrieval of the chunk. Answer only with the succinct context and nothing else.The context has to be specific to each chunk. Anthropic tried adding generic document summaries to chunks and saw very limited gains, and its evaluation of summary-based indexing showed low performance. A prompt tailored to your domain can do better than the generic one, for example by including a glossary of terms that are defined only in other documents.
Cost depends on how you order the work. Every call includes the full document, so the cookbook processes chunks document by document. The first chunk writes the document to the prompt cache, and later chunks read it at a 90% discount. On its 737-chunk dataset, input cost dropped from about $9.20 to about $2.85. The post puts the one-time cost at $1.02 per million document tokens. Contextualized chunks are longer, so check that they still fit the embedding model's input limit. If you would rather not generate context with Claude, Voyage's contextualized chunk models (voyage-context-3, voyage-context-4) take a different route: you call contextualized_embed() instead of embed().
Measure the effect with a metric such as Pass@k: does the 'golden' chunk appear in the top k results? In the cookbook, Pass@5 went from 80.92% to 88.12% with contextual embeddings. Across Anthropic's datasets, contextual embeddings cut the top-20 retrieval failure rate by 35%. Adding Contextual BM25 brought the cut to 49%.
Checkpoint 1 of 5· Check yourself
You are contextualizing 50,000 chunks from 2,000 documents and want the lowest Claude cost. How should you order the work?
The full document is cached on its first chunk and read cheaply for the rest, so you only get those savings if a document's chunks are processed one after another. Generic summaries gave very limited gains, and query-time contextualization adds latency to every search.
“This is why processing documents sequentially (rather than randomly shuffling chunks) is crucial for maximizing cache efficiency.”Source: platform.claude.com
Checkpoint 2 of 5· Exam question
A support-bot pipeline retrieves the same set of product documentation chunks for every user session and appends them to the system prompt, followed by whatever question the current user asks. The team wants to use prompt caching to cut latency and cost across the many queries that reuse this documentation. Where should they place the cache breakpoint?
Correct answer: A — On the last content block of the retrieved documentation, immediately before the user's question is added to the request.
- A. Correct. The cache breakpoint should sit on the last block that stays identical across requests. In this pipeline, that is the end of the retrieved documentation, right before the variable user question, so every query benefits from a cache hit on the stable content.
- B. Incorrect. The user's question changes on every request, so placing the breakpoint there means the cached prefix never matches between requests and no cache hits occur.
- C. Incorrect. Caching only the first sentence of the system prompt leaves the much larger, genuinely stable retrieved documentation uncached, missing almost all of the available savings.
- D. Incorrect. Caching does not require the entire request to be identical; only the cached prefix (here, the retrieved documentation) needs to match, so this pipeline is a strong caching candidate.
2.Reranking and how many chunks to pass
On a large corpus, first-stage retrieval can return hundreds of candidate chunks of uneven relevance. A reranker is a filter: it takes the query and a candidate list and scores every candidate for relevance. Anthropic's pipeline retrieves the top 150 chunks, reranks them, and passes the top 20 to Claude. Anthropic ran its tests with Cohere's reranker. Claude's documentation also lists Voyage rerankers: rerank-2.5 for highest accuracy and rerank-2.5-lite for latency and cost.
| Configuration | Failure rate | Reduction vs. baseline |
|---|---|---|
| Standard embeddings | 5.7% | — |
| Contextual Embeddings | 3.7% | 35% |
| Contextual Embeddings + Contextual BM25 | 2.9% | 49% |
| Contextual Embeddings + Contextual BM25 + reranking | 1.9% | 67% |
Reranking is not free. It adds a step at runtime, so it adds some latency even though the reranker scores chunks in parallel. Reranking more candidates improves results but costs more time and money, and reranking fewer does the reverse. The final count works the same way. More chunks raise the chance that the answer is in the prompt, but extra material can distract the model. Anthropic tried 5, 10 and 20 chunks and found 20 the best of the three. Both settings are starting points to tune on your own data, not fixed rules.
Checkpoint 3 of 5· Check yourself
A team adds a reranker over the top 500 retrieved chunks and finds p95 latency is now too high. What is the documented lever?
Anthropic describes a direct trade-off between how many candidates you rerank and latency/cost. Reranking needs the query, so it cannot move to ingestion.
“There is an inherent trade-off between reranking more chunks for better performance vs. reranking fewer for lower latency and cost.”Source: www.anthropic.com
Checkpoint 4 of 5· Exam question
A platform team is redesigning their RAG pipeline after an audit found a high rate of retrieval failures, where relevant chunks exist in the knowledge base but are not returned for matching queries. They can implement several changes before the next release. Which of the following changes are documented to measurably reduce retrieval failure rates? (Select all that apply.)(Select 3)
Correct answers: A, B, C — Prepend chunk-specific contextual summaries to chunks before generating their embeddings.; Index the same contextualized chunk text lexically with BM25 alongside the embeddings.; Add a reranking stage that rescores initial retrieval candidates before generation.
- A. Correct. Contextual embeddings, which prepend explanatory context before encoding, are documented to reduce retrieval failures on their own.
- B. Correct. Combining contextual embeddings with contextual BM25 lexical indexing further reduces retrieval failures beyond embeddings alone.
- C. Correct. Adding a reranking stage on top of contextual embeddings and BM25 produces the largest documented reduction in retrieval failures of the three combined techniques.
- D. Incorrect. Sampling temperature affects the wording and variability of the generated answer text; it has no effect on which chunks are retrieved or how retrieval failures occur.
- E. Incorrect. Binary quantization is a storage and cost optimization that reduces embedding precision; it is not a documented technique for reducing retrieval failure rates and can trade off retrieval accuracy for space.
- F. Incorrect. Disabling overlap speeds up index building but removes a safeguard against split-boundary content, which tends to increase rather than decrease retrieval failures.
3.Handing retrieved chunks to Claude with citations
The last step is getting the reranked chunks into the prompt so Claude can attribute its answer to them. The Messages API has a search_result content block for this. Claude cites search results the same way it cites web search, and each citation carries the source and title you supplied. You can supply results in two ways. Your custom retrieval tool can return them from a tool call, which suits dynamic RAG. Or you can put them directly in the user message as top-level content, which suits pre-fetched or cached chunks. No beta header is needed, and every active model except Claude Haiku 3 supports them.
{
"type": "search_result",
"source": "https://example.com/article", // Required: Source URL or identifier
"title": "Article Title", // Required: Title of the result
"content": [
// Required: Array of text blocks
{
"type": "text",
"text": "The actual content of the search result..."
}
],
"citations": {
// Optional: Citation configuration
"enabled": true // Enable/disable citations for this result
}
}These fields shape how you design your index. Store a stable identifier and a title with every chunk, because source accepts any stable string, for example an internal ID like kb://article-1234. Citations are off by default and must be enabled, and every search result in one request must use the same citation setting. Content must be text blocks only, with no images. Anthropic also suggests running evals on one generation detail: separating the generated context from the original chunk text when you pass contextualized chunks to Claude.
Checkpoint 5 of 5· Match them up
Match each search_result field to its role
Tap a term, then the definition that fits it.
source, title and content are required. citations is optional and off by default, and all results in a request must share the same setting.
“Citations are disabled by default”Source: platform.claude.com
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.Contextual Retrieval slows every search because Claude generates context per query.Why is that wrong?
Context is generated once, at ingestion, and prompt caching keeps that one-time cost low. Techniques like HyDE are the ones that add latency to each search.
Covered in Contextual retrieval: add context before you index
2.Prepending a generic summary of the document to each chunk gives the same benefit as contextual embeddings.Why is that wrong?
The context must be chunk-specific. Anthropic tested generic document summaries and saw very limited gains.
Covered in Contextual retrieval: add context before you index
3.Passing search_result blocks to Claude is enough for its answers to include citations.Why is that wrong?
Citations are disabled by default. You must set citations.enabled to true, and every result in the request must use the same setting.
Covered in Handing retrieved chunks to Claude with citations
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“Call these models with contextualized_embed() instead of embed():”
↩︎ Contextual retrieval: add context before you index“Voyage AI also offers rerankers, which take a query and a list of documents and return them ranked by relevance to the query.”
↩︎ Reranking and how many chunks to pass - 2.https://www.anthropic.com/engineering/contextual-retrievalSecondary source
“The resulting contextual text, usually 50-100 tokens, is prepended to the chunk before embedding it and before creating the BM25 index.”
↩︎ Contextual retrieval: add context before you index“the one-time cost to generate contextualized chunks is $1.02 per million document tokens”
↩︎ Contextual retrieval: add context before you index“you may be able to achieve even better results with prompts tailored to your specific domain or use case”
↩︎ Contextual retrieval: add context before you index“Combining Contextual Embeddings and Contextual BM25 reduced the top-20-chunk retrieval failure rate by 49%”
↩︎ Contextual retrieval: add context before you index“Perform initial retrieval to get the top potentially relevant chunks (we used the top 150);”
↩︎ Reranking and how many chunks to pass“Reranked Contextual Embedding and Contextual BM25 reduced the top-20-chunk retrieval failure rate by 67%”
↩︎ Reranking and how many chunks to pass“We tried delivering 5, 10, and 20 chunks, and found using 20 to be the most performant of these options”
↩︎ Reranking and how many chunks to pass“Response generation may be improved by passing it the contextualized chunk and distinguishing between what is context and what is the chunk”
↩︎ Handing retrieved chunks to Claude with citations“adding generic document summaries to chunks (we experimented and saw very limited gains)”
↩︎ Exam trap 2“There is an inherent trade-off between reranking more chunks for better performance vs. reranking fewer for lower latency and cost.”
↩︎ Checkpoint - 3.
“Processes chunks document-by-document to maximize cache hits”
↩︎ Contextual retrieval: add context before you index“Unlike techniques like HyDE (hypothetical document embeddings) that add latency to each search”
↩︎ Exam trap 1“The contextualization happens once at ingestion time, not during every query.”
↩︎ Prediction“This is why processing documents sequentially (rather than randomly shuffling chunks) is crucial for maximizing cache efficiency.”
↩︎ Checkpoint - 4.
“Use them in RAG (Retrieval-Augmented Generation) applications where Claude needs to attribute answers to your documents.”
↩︎ Handing retrieved chunks to Claude with citations“From tool calls: Your custom tools return search results, enabling dynamic RAG applications”
↩︎ Handing retrieved chunks to Claude with citations“Search results hold text only. Images and other media are not supported inside the content array.”
↩︎ Handing retrieved chunks to Claude with citations“All search results in a request must use the same setting”
↩︎ Exam trap 3“Citations are disabled by default”
↩︎ Checkpoint