What you will be able to do
- Switch an AI Search query to hybrid and explain what that costs in resources
- Use metadata filters as the main precision fix, and spot index and syntax settings that stop them working
- Configure the Databricks reranker correctly and decide when its latency is acceptable
- Recognise when semantic metadata or parsing, rather than a query setting, is limiting retrieval
1.Fix one: change the query type
Databricks' retrieval quality guide orders its fixes from highest impact and lowest effort to lowest impact and highest effort. The first is hybrid search, which combines keyword precision with semantic understanding. Use it when users search for specific terms such as product codes or technical vocabulary, or when pure vector search misses obvious keyword matches. It improves recall because it catches both semantic and keyword matches, and it improves precision for queries that contain specific terms. The change is a single query parameter.
Checkpoint 1 of 6· Fill the gap
Which value turns on combined vector and keyword retrieval in this sample?
# Enable hybrid search
results = index.similarity_search(
query_text="error code E404",
query_type=" ? " # Combines vector and keyword search
)HYBRID runs ANN and full-text together. ANN is the default that is causing the problem, FULL_TEXT drops semantic matching, and RERANK is not a query type.
Source: docs.databricks.comHybrid has costs. By default it searches all text metadata columns and returns at most 200 results. The performance guide also warns that it uses roughly twice the resources of ANN. A high-throughput application should therefore switch only when the evaluation shows hybrid actually winning. If ANN and hybrid score about the same, keep ANN.
Because it typically uses about twice the resources of ANN and can significantly reduce throughput. When keywords add nothing for your data, you pay that cost for no quality gain.
2.Fix two: metadata filtering
Suppose an agent answers a question about a 2023 Camry using a chunk from another model's manual. That is a filtering gap. Ranking can't fix it, because the wrong chunk was a valid candidate in the first place. Filtering on make, model and year, or on product version or region, can cut the search space by more than 90%, and it greatly improves precision by removing irrelevant results.
# AI Search with metadata filtering
results = index.similarity_search(
query_text="brake system maintenance",
filters='make = "Toyota" AND model = "Camry" AND year = 2023',
num_results=10
)In an agent, VectorSearchRetrieverTool with dynamic_filter=True lets the LLM pull entities out of the user's question and generate the filter string itself. Two configuration problems can make filtering fail quietly. First, the column has to be indexed: if a column is left out of Columns to index when the index is created, it can't be used as a filter. Second, the syntax depends on the endpoint type.
| Endpoint type | Filter format | Example |
|---|---|---|
| Standard | Python dictionary; the key encodes column and operator | {"price <": 200} |
| Storage-optimized | SQL-like filter string, like a WHERE clause | "price < 200" |
Checkpoint 2 of 6· Check yourself
A team adds a region column to the source table, but filtering on it in queries has no effect. What should they check first?
Only indexed columns can be used as filters, and that list is set in the index's Advanced settings.
“Only indexed columns can be returned in search results or used as filters.”Source: docs.databricks.com
Checkpoint 3 of 6· Exam question
Which of the following are valid built-in Databricks AI Search configuration levers for improving the signal quality of retrieved context, according to Databricks retrieval quality guidance? (Select all that apply.)(Select 4)
Correct answers: A, B, C, E — Enabling hybrid search (`query_type="HYBRID"`) to combine keyword and vector matching; Applying metadata filters to narrow the search space to relevant records; Adding the built-in Databricks reranker to reorder retrieved candidates; Enriching chunks with semantic metadata such as document summaries or section headers
3.Fix three: the reranker
When the right document is retrieved but ranked too low, add the reranker. After the initial retrieval, a cross-encoder re-evaluates each result against the query. It typically improves quality by about 10%, and Databricks recommends trying it for any RAG agent. It works best combined with hybrid search and filtering, not as a replacement for them.
results = index.similarity_search(
query_text = "How to create an AI Search index",
columns = ["id", "text", "parent_doc_summary", "date"],
num_results = 10,
query_type = "hybrid",
reranker=DatabricksReranker(columns_to_rerank=["text", "parent_doc_summary", "other_column"])
)Three configuration details often cause quiet problems. First, num_results sets only how many results are returned in the end. It does not set how many candidates the reranker scores. Second, the order of columns_to_rerank matters, because the reranker reads the columns in that order and considers only the first 2000 characters it finds. If a long body column is listed first, a short summary column after it may never be read. Third, the reranker can fail without raising an error. If it is temporarily unavailable, the results come back un-reranked and a warning appears in debug_info, which needs debug_level set to at least 1.
The reranker suits RAG agents, where LLM generation accounts for most of the latency, at low-to-moderate load (about 5 QPS without extra scaling). It reranks 50 results in under a second. It is not suitable for search bars that need responses under 100 msec. For those, the guide suggests serving a lightweight cross-encoder on Model Serving instead.
Checkpoint 4 of 6· Check yourself
The reranker is using a parent_doc_summary column, but it seems to have no effect on ranking. columns_to_rerank lists a long full-text column first and the summary second. What is the likely cause?
The reranker reads columns in the order listed and stops after 2000 characters. A long first column uses up that budget before the summary is read.
“The reranking calculation takes the columns in the order they are listed, and considers only the first 2000 characters it finds.”Source: docs.databricks.com
Checkpoint 5 of 6· Exam question
A team indexes PDF technical manuals into Databricks AI Search using a simple text-extraction script that concatenates raw page text. Retrieval evaluation shows the agent frequently cites the wrong numeric values because tables were flattened into unreadable strings during extraction, and section headers were lost, making chunks context-less. Which change most directly targets the root cause of this specific retrieval quality problem?
Correct answer: A — Replace the raw text-extraction script with `ai_parse_document` to produce clean, structured parsing of table and document structure before chunking.
- A. Correct. The root cause is the loss of table and document structure due to simple text concatenation. Using `ai_parse_document` produces clean, structured parsing that preserves table integrity and section hierarchy, ensuring chunks contain meaningful context and numeric values are correctly represented.
- B. Incorrect. Hybrid search improves retrieval ranking by combining keyword and semantic matching, but it does not fix the underlying problem of poorly formed chunks. If the chunks still contain garbled numeric values and lack context, hybrid search cannot reliably surface correct information.
- C. Incorrect. Adding a reranker can improve the order of retrieved chunks, but it relies on the quality of the initially retrieved content. Since the chunks are already corrupted from the extraction process, the reranker would be reordering chunks with inaccurate numeric values, not addressing the root cause.
- D. Incorrect. Increasing the number of results provides more chunks for cross-checking, but this does not resolve the fundamental issue of each chunk containing flattened, context-less text. Without proper extraction, the agent would still encounter malformed numeric values across all retrieved chunks.
4.When the configuration is fine but the signal isn't
If every strategy scored low on the evaluation, the problem is in the data. The guide's advice is not to spend effort fine-tuning chunk size. It recommends higher-impact steps instead: extract metadata so that filtering is possible, parse documents with ai_parse_document, and add semantic context such as document summaries and section headers. Poor parsing, such as missing tables or broken formatting, directly lowers retrieval quality. Semantic metadata also connects back to configuration. If you store it in separate columns, a query only uses it when it is configured to: semantic columns have to be listed in columns_to_rerank, and keyword-only columns are matched through hybrid (full-text) search.
Checkpoint 6 of 6· Check yourself
Document summaries are stored in a separate semantic metadata column, but retrieval ignores them. Which query setting should the team configure?
Semantic metadata stored in separate columns is only used when the reranker is told to consider those columns through columns_to_rerank.
“For semantic metadata: Use reranking with columns_to_rerank parameter to consider these columns.”Source: docs.databricks.com
Sources1
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.Raising num_results gives the reranker more candidates to choose from.Why is that wrong?
num_results controls only how many results are returned in the end. It does not change how many results the reranker scores.
Covered in Fix three: the reranker
2.Reranking is the strongest fix when the agent retrieves chunks from the wrong product or version.Why is that wrong?
Wrong-scope chunks are a filtering problem. Filtering removes them before ranking, and the guide calls it the biggest lever for retrieval quality.
Covered in Fix two: metadata filtering
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“Improves recall by catching both semantic and keyword matches.”
↩︎ Fix one: change the query type“Can reduce search space by 90%+.”
↩︎ Fix two: metadata filtering“Works best when combined with techniques like hybrid search and filtering.”
↩︎ Fix three: the reranker“Real-time search bars requiring <100 msec latency.”
↩︎ Fix three: the reranker“Poor parsing (missing tables, broken formatting) directly impacts retrieval quality.”
↩︎ When the configuration is fine but the signal isn't“For semantic metadata: Use reranking with columns_to_rerank parameter to consider these columns.”
↩︎ When the configuration is fine but the signal isn't“Dramatically improves precision by eliminating irrelevant results.”
↩︎ Exam trap 2“This is your biggest lever for retrieval quality.”
↩︎ Prediction - 2.
“By default, hybrid search includes all text metadata columns and returns a maximum of 200 results.”
↩︎ Fix one: change the query type“The reranker typically improves quality by approximately 10% but adds latency.”
↩︎ Fix three: the reranker“num_results is the final number of results to return. This does not affect the number of results used for reranking.”
↩︎ Exam trap 1“The reranking calculation takes the columns in the order they are listed, and considers only the first 2000 characters it finds.”
↩︎ Checkpoint - 3.
“Hybrid search typically uses about twice as many resources as ANN and can significantly reduce throughput.”
↩︎ Fix one: change the query type - 4.
“standard endpoints use a Python dictionary, and storage-optimized endpoints use a SQL-like filter string.”
↩︎ Fix two: metadata filtering
Also cited
“Only indexed columns can be returned in search results or used as filters.”
↩︎ Checkpoint