CertSafari
    Anthropic Claude Certified Developer – Professional (CCDV-P)· Lessons

    Domain 2 · Lesson 8/25

    Retrieval architecture and grounding for production Claude apps

    Retrieval and Knowledge Grounding at Scale

    18 min read
    5% of exam
    5 sources
    Published 19 Nov 2026
    Docs as of 11 Nov 2026

    What you will be able to do

    • Choose between full-context prompting, indexed RAG, and agentic search for a given corpus size and change rate
    • Repair chunking failures where the retrieved passage lacks the heading, entity, or defined term it depends on
    • Combine BM25 with embeddings, then rerank, to fix exact-identifier misses without flooding the prompt
    • Use citations, search result blocks, and uncertainty instructions to produce answers that can be checked
    • Find which pipeline stage caused a bad answer and apply the fix at that stage

    Key concept

    Fix the stage that lost the information — A grounded answer can only use text that survives every stage: chunking, indexing, retrieval, selection, then generation. When a fact is lost at an early stage, no later stage can bring it back. Diagnose which stage dropped it and change that stage.

    1.Full context, indexed RAG, or agentic search

    Every knowledge-grounded application first has to decide how source text reaches the model. There are three options, and the choice depends on how big the corpus is and how fast it changes. The smallest case needs no retrieval at all. If a knowledge base is under about 200,000 tokens, roughly 500 pages, you can put the whole thing in the prompt. Prompt caching then makes repeated requests cheaper and faster. Below that size, a retrieval pipeline only adds ways to miss something.

    Indexed RAG is the usual answer once the corpus is too large for the context window. The work happens ahead of time. You split the corpus into chunks of a few hundred tokens, embed them into vectors, and store them in a vector database. At query time the system finds the chunks most similar to the question and adds them to the prompt. This scales to very large corpora because the expensive steps run once, before any question arrives.

    That same advantage is also the limitation: an index is a snapshot from its last build. Take a monorepo that changes hundreds of times a day. With a nightly rebuild, a function renamed this morning is still listed under its old name. Rebuilding every 30 minutes only shrinks the window. Ranking recent chunks higher still ranks within the same old snapshot. Re-chunking changes how the text is cut, not how old it is. The direct fix is agentic search: give the model tools such as glob, grep, and file-read over the live files, so retrieval happens when the question is asked.

    Agentic search has its own costs. Runtime exploration is slower than reading a precomputed index. An agent with poorly designed tools or vague guidance wastes context on dead ends or misses the key file. The step-by-step exploration does have a benefit: each result tells the agent where to look next. A practical middle ground is a hybrid. Load stable, known-relevant material up front and let the agent explore the rest. Bounding how long that exploration runs is a latency question, covered in latency engineering.

    Matching the retrieval architecture to the corpus
    ArchitectureWins whenMain cost
    Whole corpus in the prompt (with prompt caching)Knowledge base under about 200,000 tokensDoes not scale past the context window
    Indexed RAG (chunks, embeddings, vector database)Large corpus that changes slowly compared with the rebuild scheduleAnswers can be stale; context is lost at chunk boundaries
    Agentic search (tools that fetch content at runtime)Content changes constantly, or the answer takes several dependent lookupsSlower, and needs good tools and guidance

    Checkpoint 1 of 9· Match them up

    Match each retrieval architecture to the knowledge-grounded application it fits best.

    Tap a term, then the definition that fits it.

    Checkpoint 2 of 9· Exam question

    Russet Learning runs an internal engineering assistant that answers developer questions about its monorepo, which receives hundreds of commits a day. The assistant retrieves from a vector index that is rebuilt nightly. Developers report that answers cite functions that were renamed earlier the same morning, and the logs show the retrieved chunks contain only the pre-rename names. Engineers have noted that a full re-embedding of the repository is slow and costly, and a few asked questions within minutes of merging their own changes and still got stale answers. The team accepts a few extra seconds of latency per question if answers reflect the code as it currently stands. Which change most directly fixes the staleness?

    Sources123

    2.Chunking that keeps the context a passage needs

    Chunking is where context most often disappears. Consider a chunk from an SEC filing that says "The company's revenue grew by 3% over the previous quarter." On its own, it does not say which company or which quarter. A question about ACME's Q2 revenue may never retrieve it. If it is retrieved, the model still cannot tell what it refers to. Contracts fail the same way. A fixed 200-token window can catch the dollar figure of an indemnification cap but leave the clause heading and the defined term in neighbouring chunks.

    Fixed-size chunks will not solve this by getting smaller. A 100-token chunk with no overlap focuses each embedding on a single statement, but it removes even more of the surrounding text the statement depends on. Three settings decide what each chunk contains: chunk size, where the boundaries fall, and how much neighbouring chunks overlap. For structured documents, cut along the document's own structure. Split contracts at section and clause boundaries, with a modest overlap, so each passage keeps its title and the whole clause.

    A second fix separates the unit you search from the unit you send to the model. Index small chunks so matching stays precise. Then, before generation, replace each hit with its enclosing section, so the model sees the heading and definitions with the figure. Neither a larger embedding model nor a reranker can do this. Both work only on the text inside the chunk, and they cannot add a heading that was cut away. A generation-time instruction to "cite the clause title" has the same problem: the title is not in the model's context.

    Contextual Retrieval puts the missing context back into each chunk before indexing. Claude reads the whole document and writes a 50–100 token description that places one chunk within it. That text is prepended to the chunk before embedding and before building the BM25 index. The SEC chunk becomes "This chunk is from an SEC filing on ACME corp's performance in Q2 2023…". With prompt caching, the document is cached once and reused for every chunk, so this preprocessing is cheap. Contextual embeddings alone cut top-20 retrieval failures by 35%. The method helped with every embedding model tested, though some models gained more than others.

    Chunk structure matters at citation time too. With custom content documents and search result blocks, the text blocks you provide are the units Claude can cite. Splitting long content into logical blocks gives Claude finer citation boundaries.

    Checkpoint 3 of 9· Check yourself

    A contract-review RAG system splits documents into fixed 200-token chunks. Retrieved chunks often hold an indemnification cap's dollar figure, but the clause heading and defined term are in neighbouring chunks. Answers then assign the cap to the wrong clause. Which change fixes the stage that lost the context?

    Checkpoint 4 of 9· Exam question

    Quarry Grocers runs a legal-research tool for its procurement team that answers questions about supplier contracts. The system splits every contract into fixed 200-token chunks. Answers about indemnification caps are often wrong: the retrieved chunk contains the dollar figure, but the clause heading and the defined term it refers to sit in neighboring chunks. A review of 50 failed answers shows the correct chunk ranked first in nearly every case. The team already runs generation at temperature 0, and its prompt asks the model to name the clause title and defined terms, yet the model replies that they are not stated. Contracts follow a consistent layout of numbered sections with headings. Select TWO changes that best address this.(Select 2)

    0 of 2 selected

    Sources12

    3.Hybrid search and reranking

    Dense embeddings capture meaning, but they can miss exact strings. Ask a support index about "Error code TS-999". An embedding model may return content about error codes in general and skip the one document that contains TS-999. Error codes, version strings, part numbers, and other identifiers are exactly the tokens that semantic similarity tends to blur.

    BM25 is the lexical counterpart. It is a ranking function based on TF-IDF that scores exact word and phrase matches. It adjusts for document length and caps how much repeated terms count, so common words do not dominate. A hybrid pipeline runs both retrievers over the same chunks. BM25 takes the top chunks by exact match, embeddings take the top chunks by meaning, and rank fusion merges the two lists and removes duplicates. Contextual embeddings plus contextual BM25 cut top-20 retrieval failures by 49%.

    Several tempting fixes stay on the dense-only path and fail. Lowering the similarity threshold admits more borderline chunks, but the exact-match document still does not rank. Raising top-k to 200 and letting the model pick hands it a pile of mostly irrelevant text. More chunks raise the chance the right one is included, but extra information distracts the model, so there is a limit. A system-prompt rule to "prefer chunks containing the exact code" acts at generation time. It cannot help when retrieval never returned the chunk.

    Reranking resolves the conflict between wide recall and a short prompt. First, retrieve broadly. Anthropic's experiments took the top 150 candidates. A reranking model then scores each candidate against the query, and only the best few go to the model; they kept the top 20. Because the model processes less text, answers get better and cost and latency go down. Adding reranking to Contextual Retrieval raised the reduction in failed retrievals from 49% to 67%. Tool-based retrieval follows the same rule: return only the most relevant results so the context does not overflow.

    Where each hybrid-pipeline component helps
    ComponentCatchesMisses on its own
    Dense embeddingsParaphrases and matches by meaningExact identifiers and technical terms
    BM25Exact tokens such as error codes and version stringsParaphrased questions with no shared words
    Rank fusionCombines both lists without duplicatesDoes not shorten what reaches the prompt
    RerankerNarrows a wide candidate set (top 150) to a short context (top 20)Cannot recover a chunk that neither retriever returned

    Checkpoint 5 of 9· Check yourself

    Users of a support assistant search for "Error code TS-999". The retriever uses dense embeddings only. It returns general articles about error codes, and the one document that mentions TS-999 never appears. Which change fixes this miss?

    Checkpoint 6 of 9· Check yourself

    After adding hybrid search, the relevant chunk usually shows up in the fused top 150, often somewhere between rank 50 and rank 150. Sending all 150 chunks makes answers worse, slower, and more expensive. What should the pipeline do?

    Checkpoint 7 of 9· Exam question

    Saffron Energy runs a product-support bot for its smart-meter and inverter customers, retrieving manuals and release notes using only dense embeddings. For queries like "error E-4021 on firmware 3.2.1", the top results discuss generic firmware errors. A literal search of the corpus for "E-4021" confirms the correct release-note page exists, yet it ranks far below position 200 and never reaches the model. In testing, a looser similarity cutoff only admitted more generic firmware pages. Token spend per answer is already near budget, and the system prompt already tells the model to cite exact codes. Other phrasings retrieve well, and the team wants to keep that behaviour. Select TWO changes that address this retrieval failure.(Select 2)

    0 of 2 selected

    Sources12

    4.Grounding answers: citations, uncertainty, and confidence signals

    Retrieval decides what the model can see. Grounding decides whether the answer can be checked against it. Citations make it checkable. When citations are enabled on documents, the response is split into text blocks. Each block can carry citations that point to the exact passage supporting it: character ranges for plain text, page ranges for PDFs, and block indices for custom content. Scanned PDFs without extractable text cannot be cited. The cited_text field does not count toward output tokens.

    For RAG output, wrap each retrieved passage in a search result content block rather than pasting it as plain text. Each citation then carries the source and title you set, so an answer traces back to a URL or an internal ID such as kb://article-1234. Search results can come from a custom tool's result or sit directly in the user message. Citations are disabled by default, and every search result in a request must use the same setting.

    The search_result block: source and title travel with every citationjson
    {
      "type": "search_result",
      "source": "https://example.com/article", // Required: Source URL or identifier
      "title": "Article Title", // Required: Title of the result
      "content": [
        // Required: Array of text blocks
        {
          "type": "text",
          "text": "The actual content of the search result..."
        }
      ],
      "citations": {
        // Optional: Citation configuration
        "enabled": true // Enable/disable citations for this result
      }
    }

    Prompt format can quietly turn citations off. A prompt that asks for only a JSON object still succeeds, but the response may contain few or no citations. To get a citation for each extracted value, ask for one field: value line per field and parse the lines into structured data in your own code.

    Saying "I don't know" has to be designed in. Give Claude explicit permission to admit uncertainty. For example: "If the report lacks necessary information, say 'I don't have enough information to confidently assess this.'" Tell it to use only the provided documents, not its general knowledge. For long documents, ask it to extract word-for-word quotes first, and to state "No relevant quotes found." when there are none. Handle a search that returns nothing the same way. Return a plain text block such as No results found. instead of raising an error, so Claude can explain the empty result and the conversation continues.

    Confidence signals come from checks you can see, not from the model's tone. Three are available:

    • Whether each claim has a citation at all.
    • Whether a claim survives a verification pass, where Claude looks for a supporting quote for each claim and removes any claim it cannot support.
    • Whether answers agree across best-of-N runs, since inconsistency across outputs can indicate hallucination.

    A claim with no citation, removed during verification, or that changes between runs is a low-confidence claim. Show it that way to users.

    Checkpoint 8 of 9· Fill the gap

    You wrap each retrieved passage in a search result block so that every citation traces back to a URL or an internal ID such as kb://article-1234. Which field name completes the block?

    {
      "type": "search_result",
      " ? ": "https://example.com/article", // Required: Source URL or identifier
      "title": "Article Title", // Required: Title of the result
      "content": [
        // Required: Array of text blocks
        {
          "type": "text",
          "text": "The actual content of the search result..."
        }
      ],
      "citations": {
        // Optional: Citation configuration
        "enabled": true // Enable/disable citations for this result
      }
    }

    Sources415

    5.Diagnosing retrieval-quality failures by stage

    A wrong grounded answer is a symptom. The cause sits in one stage of the pipeline, and each stage leaves its own evidence. Start by checking whether the right passage reached the model's context. If it never did, the fault is upstream: chunking or retrieval. If it did and the answer is still wrong or unsupported, look at selection and generation. A team that tunes prompts for a retrieval miss, or swaps embedding models for a generation error, makes changes that cannot work.

    Symptoms, likely stage, and the fix that acts on that stage
    StageSymptomFix at this stage
    Chunking / indexingRight chunk retrieved, but it lacks the entity, heading, or defined termStructure-aware boundaries with overlap; contextual chunks; pass the enclosing section
    Index freshnessAnswers cite names or facts that changed after the last buildAgentic search with tools that read live content at runtime
    RetrievalRelevant chunk not in the top 20 (exact identifiers missed)Add BM25 alongside embeddings and fuse the rankings
    Selection / rerankingRight chunk retrieved but crowded out, or the prompt is bloatedRerank a wide candidate set down to a short top-K
    Generation / groundingPassage was in context, but the claim is uncited or unsupportedEnable citations, check every claim has one, verify claims against quotes

    Measure retrieval on its own, apart from answer quality. Anthropic's experiments used 1 minus recall@20: the share of relevant documents that fail to appear in the top 20 retrieved chunks. That number tells you whether a change to chunking, embeddings, BM25, or the reranker helped retrieval, without generation getting in the way. Run it on a labelled question set before and after every pipeline change. Building that harness belongs to evaluation system design.

    Check grounding on its own as well. The API does not guarantee that every extracted field has a value or that every value has a citation, so check both in code. A citation that exists can still fail to support its sentence. Compare cited_text against the claim it is attached to, or have Claude find a supporting quote for each claim and remove claims it cannot support.

    Checkpoint 9 of 9· Check yourself

    Request logs for a wrong answer show that the chunk holding the correct quarterly revenue figure was retrieved and placed in the prompt. Even so, the answer states a different figure that appears nowhere in that chunk. Which change acts on the stage that failed?

    Sources452

    Sources

    (5)

    Every claim above is drawn from one of these pages, quoted as it was written on the date shown.

    1. 1.
      “tools fetch content at runtime, and Claude cites it in the response”
      ↩︎ Full context, indexed RAG, or agentic search
      “Break long content into logical text blocks to give Claude finer citation boundaries”
      ↩︎ Chunking that keeps the context a passage needs
      “Result limits: Return only the most relevant results to avoid context overflow”
      ↩︎ Hybrid search and reranking
      “each citation carries the source and title you provided”
      ↩︎ Grounding answers: citations, uncertainty, and confidence signals
      “when a search fails or returns nothing, return a plain text block describing the outcome”
      ↩︎ Grounding answers: citations, uncertainty, and confidence signals
      “tools fetch content at runtime, and Claude cites it in the response”
      ↩︎ Exam trap 4
      “Returning search results from your custom tools lets you build dynamic RAG applications: tools fetch content at runtime”
      ↩︎ Checkpoint
    2. 2.
      “you can just include the entire knowledge base in the prompt that you give the model, with no need for RAG or similar methods”
      ↩︎ Full context, indexed RAG, or agentic search
      “The choice of chunk size, chunk boundary, and chunk overlap can affect retrieval performance”
      ↩︎ Chunking that keeps the context a passage needs
      “The resulting contextual text, usually 50-100 tokens, is prepended to the chunk before embedding it and before creating the BM25 index.”
      ↩︎ Chunking that keeps the context a passage needs
      “Response generation may be improved by passing it the contextualized chunk and distinguishing between what is context and what is the chunk.”
      ↩︎ Chunking that keeps the context a passage needs
      “Whereas Contextual Retrieval improves performance across all embedding models we tested, some models may benefit more than others.”
      ↩︎ Chunking that keeps the context a passage needs
      “It's particularly effective for queries that include unique identifiers or technical terms.”
      ↩︎ Hybrid search and reranking
      “Combine and deduplicate results from (3) and (4) using rank fusion techniques;”
      ↩︎ Hybrid search and reranking
      “Reranking is a commonly used filtering technique to ensure that only the most relevant chunks are passed to the model.”
      ↩︎ Hybrid search and reranking
      “This method can reduce the number of failed retrievals by 49% and, when combined with reranking, by 67%.”
      ↩︎ Hybrid search and reranking
      “However, more information can be distracting for models so there's a limit to this.”
      ↩︎ Hybrid search and reranking
      “measures the percentage of relevant documents that fail to be retrieved within the top 20 chunks”
      ↩︎ Diagnosing retrieval-quality failures by stage
      “traditional RAG solutions remove context when encoding information, which often results in the system failing to retrieve the relevant information from the knowledge base.”
      ↩︎ Key concept
      “While embedding models excel at capturing semantic relationships, they can miss crucial exact matches.”
      ↩︎ Exam trap 1
      “it can lead to problems when individual chunks lack sufficient context.”
      ↩︎ Exam trap 2
      “adding generic document summaries to chunks (we experimented and saw very limited gains)”
      ↩︎ Prediction
      “Chunk boundaries: Consider how you split your documents into chunks. The choice of chunk size, chunk boundary, and chunk overlap can affect retrieval performance”
      ↩︎ Checkpoint
    3. 3.
      “runtime exploration is slower than retrieving pre-computed data”
      ↩︎ Full context, indexed RAG, or agentic search
      “Without proper guidance, an agent can waste context by misusing tools, chasing dead-ends, or failing to identify key information.”
      ↩︎ Full context, indexed RAG, or agentic search
      “the most effective agents might employ a hybrid strategy, retrieving some data up front for speed, and pursuing further autonomous exploration at its discretion”
      ↩︎ Full context, indexed RAG, or agentic search
    4. 4.
      “Citations return the exact passages that support each claim, so you can verify answers and surface sources to your users.”
      ↩︎ Grounding answers: citations, uncertainty, and confidence signals
      “PDFs that are scans of documents and do not contain extractable text are not citable.”
      ↩︎ Grounding answers: citations, uncertainty, and confidence signals
      “Check that each field has a value and at least one citation.”
      ↩︎ Diagnosing retrieval-quality failures by stage
      “A request with such a prompt succeeds, but the response might contain few or no citations.”
      ↩︎ Exam trap 3
    5. 5.
      “Explicitly give Claude permission to admit uncertainty. This technique can drastically reduce false information.”
      ↩︎ Grounding answers: citations, uncertainty, and confidence signals
      “Inconsistencies across outputs could indicate hallucinations.”
      ↩︎ Grounding answers: citations, uncertainty, and confidence signals
      “You can also have Claude verify each claim by finding a supporting quote after it generates a response.”
      ↩︎ Diagnosing retrieval-quality failures by stage
      “have Claude verify each claim by finding a supporting quote after it generates a response”
      ↩︎ Checkpoint

    Ready to test yourself?

    Practise Anthropic Claude Certified Developer – Professional (CCDV-P) in quiz mode.

    Spotted a mistake, or was something unclear? Tell us.