CertSafari

    Free Databricks Certified Generative AI Engineer Associate Sample Questions

    35 free sample questions from our bank of 349+, covering every exam domain, with answers and detailed explanations. Updated August 2026.

    Domain 1: Design Applications

    Subdomain 1.2: Select model tasks to accomplish a given business requirement

    1.A legal operations team wants to pull specific fields (contract start date, renewal term, counterparty name) out of thousands of scanned PDF contracts and load the results into a Delta table with a fixed schema. Which model task should the application be built around?

    1. A.Structured information extraction against a defined schema
    2. B.Open-ended text generation to draft a plain-language summary of each contract
    3. C.Multi-turn chat that lets a user ask follow-up questions about one contract at a time
    4. D.Multi-class classification that assigns each contract to a single category label
    Show answer & explanation

    Correct answer: AStructured information extraction against a defined schema

    • A. Pulling named fields such as dates, terms, and party names into a fixed schema is exactly what structured information extraction (e.g. `ai_extract`) is designed for, and it maps directly onto a Delta table with fixed columns.
    • B. Generation would produce free-form prose summaries, which do not give the fixed, queryable columns (start date, term, counterparty) the team needs for the Delta table.
    • C. A chat interface answers ad hoc questions interactively but does not batch-produce structured field values across thousands of documents for a table load.
    • D. Classification assigns each document to one of a set of predefined labels, which does not recover the specific field values the team needs to populate table columns.

    Subdomain 1.1: Design a prompt that elicits a specifically formatted response

    2.A developer's prompt asks an LLM to "respond with only the JSON object, nothing else," but the model still occasionally prepends a sentence like "Here is the JSON:" before the object, which breaks a strict `json.loads()` call downstream. The developer wants to keep using plain JSON without switching to a schema-validation library. Which prompt adjustment is most likely to eliminate the extra text while keeping the fix simple?

    1. A.Instruct the model to wrap the JSON output in specific delimiters such as triple backticks, then have the downstream code extract only the text between those delimiters
    2. B.Remove the JSON formatting instruction entirely so the model can answer more naturally and the code can search the free text for key phrases
    3. C.Ask the model to repeat the instruction back before answering, confirming that it understands only JSON should be returned
    4. D.Shorten the prompt to a single word like "Extract" so the model has less room to add conversational filler around the answer
    Show answer & explanation

    Correct answer: AInstruct the model to wrap the JSON output in specific delimiters such as triple backticks, then have the downstream code extract only the text between those delimiters

    • A. Asking the model to enclose its output in a distinctive delimiter such as triple backticks gives the downstream code a reliable anchor to extract exactly the JSON substring, even if the model still adds a stray lead-in sentence outside the delimiters.
    • B. Dropping the JSON instruction removes the format constraint altogether, making the response less structured and harder to parse, which is the opposite of what the pipeline needs.
    • C. Having the model restate the instruction adds extra conversational text to the response and does not stop it from also adding a lead-in sentence before the JSON object.
    • D. A one-word prompt removes the context the model needs to know what fields to extract and does not specifically address the lead-in text problem; it is likely to make the output less reliable, not more.

    Subdomain 1.5: Define and order tools that gather knowledge or take actions for multi-stage reasoning

    3.A generative AI engineer is building a tool-calling agent for a claims-processing use case. The agent has three tools: `lookup_claim_details`, `check_policy_coverage`, and `issue_payment`. Payments must never be issued for claims that fail coverage verification. How should the engineer define and order these tools for the agent's multi-stage reasoning loop?

    1. A.Constrain `issue_payment` so it only runs after `check_policy_coverage` approves the claim, and write tool descriptions that lead the agent to call lookup, then coverage check, then payment in that order.
    2. B.Let the agent call `issue_payment` first as the main objective, then invoke `lookup_claim_details` and `check_policy_coverage` afterward purely to produce an audit log of the completed transaction.
    3. C.Give all three tools identical generic descriptions so the LLM treats them as interchangeable, and let it invoke `issue_payment` and `check_policy_coverage` in parallel to cut latency.
    4. D.Drop `check_policy_coverage` from the tool set entirely and depend on a human reviewer to catch any improperly paid claims after the payment has already settled.
    Show answer & explanation

    Correct answer: AConstrain `issue_payment` so it only runs after `check_policy_coverage` approves the claim, and write tool descriptions that lead the agent to call lookup, then coverage check, then payment in that order.

    • A. Constraining the payment tool to run only after coverage approval, and describing the tools so the agent naturally sequences lookup, then verification, then action, matches Databricks guidance that consequential actions in a multi-stage reasoning loop should be gated behind the verification step they depend on. This prevents the model from reaching an irreversible action tool before the required check has succeeded.
    • B. Calling the payment tool before verifying coverage defeats the purpose of the coverage check, since the payment would already be issued by the time the audit-log calls run. This ordering allows an unapproved claim to be paid, which is exactly the failure mode the design must prevent.
    • C. Identical generic tool descriptions remove the signal the agent needs to reason about which tool serves which purpose, and running the payment and coverage-check tools in parallel means the payment can complete before the coverage result is even known. This creates a race condition that can pay out non-covered claims.
    • D. Removing the coverage-check tool eliminates the automated gate entirely, so every claim would be paid regardless of eligibility, with correctness pushed onto a reviewer who only sees the claim after money has already moved. This does not satisfy the requirement that payments never occur for claims failing verification.

    Domain 2: Data Preparation

    Subdomain 2.1: Apply a chunking strategy for a given document structure and model constraints

    4.An engineer builds a RAG pipeline over long onboarding guides. During testing, they notice that a key instruction spanning the end of one chunk and the beginning of the next is never retrieved in full: the retriever returns one chunk or the other, but neither chunk alone contains the complete instruction. Which adjustment to the chunking configuration most directly addresses this failure?

    1. A.Configure adjacent chunks to overlap by a set number of tokens so boundary content appears in more than one chunk.
    2. B.Increase the total number of chunks retrieved at query time so the retriever returns more candidate chunks per query.
    3. C.Switch the embedding model to one trained specifically on the onboarding guide's domain vocabulary.
    4. D.Sort the chunks by document order before indexing so neighboring chunks sit next to each other in the vector index.
    Show answer & explanation

    Correct answer: AConfigure adjacent chunks to overlap by a set number of tokens so boundary content appears in more than one chunk.

    • A. Adding overlap between adjacent chunks means content near a boundary is duplicated into both chunks, so an instruction that straddles the boundary will appear in full inside at least one chunk and can be retrieved intact.
    • B. Returning more candidate chunks increases the chance both neighboring chunks are retrieved together, but neither chunk individually contains the complete instruction, so the response context is still fragmented across two partial chunks.
    • C. A domain-tuned embedding model can improve semantic matching for retrieval, but it does not change where chunk boundaries fall, so the instruction would still be split across two chunks.
    • D. Physical storage order in the vector index does not affect which chunks are returned for a similarity search, so this does not help the retriever surface the complete instruction.

    Subdomain 2.4: Define operations and sequence to write given chunked text into Delta Lake tables in Unity Catalog

    5.A team writes new batches of chunked text into their Unity Catalog Delta table once per night via a scheduled job, and they want the AI Search index to refresh once after each nightly batch completes rather than continuously watching the table for changes. Which index configuration achieves this?

    1. A.Create the Delta Sync Index with `pipeline_type="TRIGGERED"` so the index only syncs when explicitly triggered or scheduled after each nightly write completes.
    2. B.Create the Delta Sync Index with `pipeline_type="CONTINUOUS"` so the index runs on a dedicated always-on cluster that watches the Delta table and syncs immediately as each night's rows are committed.
    3. C.Create a Direct Vector Access Index and call its upsert API from the nightly job so the index updates in lockstep with the batch write, bypassing the Delta table sync mechanism entirely.
    4. D.Create the Delta Sync Index without specifying `pipeline_type` and rely on the default hourly cron schedule, since Delta Sync indexes always poll the source table once per hour regardless of configuration.
    Show answer & explanation

    Correct answer: ACreate the Delta Sync Index with `pipeline_type="TRIGGERED"` so the index only syncs when explicitly triggered or scheduled after each nightly write completes.

    • A. A triggered pipeline type syncs the index on demand or on a schedule the team controls, which matches a workflow where new data lands once nightly and the index only needs to refresh after that batch is written.
    • B. A continuous pipeline type keeps compute running at all times to sync near-instantly as changes occur, which is unnecessary and more costly for a workload that only produces new data once per night.
    • C. A direct access index requires the application to manage embeddings and upserts itself via API calls rather than syncing from a Delta table, which abandons the Delta table-driven sync workflow the team already has in place.
    • D. A Delta Sync Index does not fall back to a fixed hourly schedule when `pipeline_type` is unspecified; the pipeline type must be explicitly chosen to control whether syncing is triggered or continuous.

    Subdomain 2.5: Identify needed source documents that provide necessary knowledge and quality for a given RAG application

    6.A team is preparing a source corpus by scraping publicly available community forum posts to supplement an internal knowledge base for a RAG assistant. Some forum posts contain personal contact details and occasional offensive language. Before these documents are added as knowledge sources, what should the team do to protect application quality and safety?

    1. A.Run the scraped documents through classifiers to detect and filter out personal information and harmful language before ingestion
    2. B.Ingest the forum posts as-is and rely on the LLM's guardrails at inference time to avoid repeating any sensitive content
    3. C.Shorten the chunk size used for the forum posts so any sensitive text is split across multiple smaller chunks
    4. D.Store the forum posts in a separate Delta table so they are queried less frequently than the internal knowledge base
    Show answer & explanation

    Correct answer: ARun the scraped documents through classifiers to detect and filter out personal information and harmful language before ingestion

    • A. Applying classifiers to detect personal information and harmful language during source preparation directly prevents sensitive or offensive content from entering the corpus at all. This matches the guidance that sensitive information and harmful language in source documents should be filtered using classifiers before ingestion.
    • B. Relying on inference-time guardrails does not stop the retriever from surfacing the sensitive content as retrieved context, and the guardrail could still fail to catch every instance. The underlying risk originates in the source data, so filtering needs to happen during preparation rather than after retrieval.
    • C. Splitting sensitive text across smaller chunks does not remove personal information or offensive language from the corpus; the content is still present and retrievable, just fragmented. Chunk size affects retrieval granularity, not content safety.
    • D. Placing the forum posts in a separate table that is queried less often reduces but does not eliminate the chance that sensitive or harmful content is retrieved and surfaced to users. The unfiltered content remains available as a knowledge source.

    Subdomain 2.6: Use tools and metrics to evaluate retrieval performance

    7.A technical support search tool must return the single correct troubleshooting article at the very top of the results, since agents only read the first couple of results before acting. Which metric best evaluates this requirement?

    1. A.Precision@k evaluated at a small k, such as precision@3, to confirm that nearly every top result is relevant
    2. B.Recall@k evaluated at a large k, such as recall@50, to confirm that all relevant articles exist somewhere in the results
    3. C.Average relevance score computed across the entire retrieved candidate set, not just the top results
    4. D.Relevance distribution across all graded buckets, to check the overall spread of result quality
    Show answer & explanation

    Correct answer: APrecision@k evaluated at a small k, such as precision@3, to confirm that nearly every top result is relevant

    • A. Precision at a small k directly measures whether the handful of results an agent actually reads are relevant, which matches a scenario where exact top-of-list accuracy matters more than exhaustive coverage.
    • B. Recall at a large k answers whether relevant articles exist anywhere in a broad candidate pool, which does not address whether the top one or two results the agent will actually read are correct.
    • C. An average relevance score across the whole candidate set dilutes the signal from the top positions with lower-ranked results the agent will never see, so it does not isolate top-of-list accuracy.
    • D. A relevance distribution describes how graded scores are spread across all retrieved results, which is a diagnostic overview rather than a targeted measure of whether the very top result is correct.

    Subdomain 2.7: Design retrieval systems using advanced chunking strategies

    8.During evaluation, a RAG engineer notices that answers referencing information near a chunk boundary are frequently incomplete, because the fact needed to answer the question is split across two adjacent chunks. Which change to the chunking configuration best addresses this specific problem?

    1. A.Increase the number of chunks retrieved per query so more candidate chunks reach the language model.
    2. B.Configure the chunker to carry a window of trailing sentences into the start of the next chunk.
    3. C.Switch from a paragraph-based splitter to a fixed-character splitter to produce more uniform boundaries.
    4. D.Reduce the chunk size so each chunk covers a narrower span of the source document than before.
    Show answer & explanation

    Correct answer: BConfigure the chunker to carry a window of trailing sentences into the start of the next chunk.

    • A. Retrieving more chunks increases the chance that a second chunk with the missing fact is also returned, but it does not guarantee it, and it adds noise and cost to the prompt without directly fixing the boundary-splitting issue.
    • B. Carrying a window of trailing sentences from one chunk into the next, known as chunk overlap, ensures continuity across the boundary so a fact split between two chunks is likely to appear intact in at least one of them.
    • C. Switching to a fixed-character splitter changes where boundaries fall but does not add continuity across them, so facts can still be split between chunks just as before, only at different points in the text.
    • D. Shrinking the chunk size increases the number of boundaries in the document, which makes it more likely, not less, that a given fact will end up split across two separate chunks.

    Subdomain 2.8: Explain the role of re-ranking in the information retrieval process

    9.During retrieval quality evaluation, an engineer runs the same set of test queries through the pipeline twice — once with the reranker disabled and once with it enabled — and compares DCG@10 scores for each query type. For one query type, enabling the reranker raises DCG@10 significantly with only a modest latency increase; for another query type, it makes almost no difference. What is the most appropriate way to use this evaluation result?

    1. A.Enable the reranker for the query types where it measurably improves DCG@10 within the latency budget, and skip it where the gain doesn't justify the cost
    2. B.Enable the reranker everywhere, since any nonzero DCG@10 improvement on any query type justifies applying it uniformly across all traffic
    3. C.Disable the reranker everywhere, since a query type with no DCG@10 improvement proves the reranker doesn't help this retrieval pipeline overall
    4. D.Discard the DCG@10 results entirely, since re-ranking quality can only be judged using recall@k rather than any ranking-order-sensitive metric
    Show answer & explanation

    Correct answer: AEnable the reranker for the query types where it measurably improves DCG@10 within the latency budget, and skip it where the gain doesn't justify the cost

    • A. Comparing DCG@10 with and without the reranker per query type is exactly how retrieval quality evaluation is meant to guide a rollout decision: enable reranking where it delivers a real, latency-affordable quality gain, and leave it off where the benefit doesn't clear that bar. This targeted approach makes the best use of the evaluation data collected.
    • B. Applying the reranker uniformly ignores the finding that one query type saw almost no benefit; enabling it there still adds latency cost without a corresponding quality payoff. The evaluation data specifically shows the impact varies by query type, so a blanket decision discards that signal.
    • C. One query type showing little improvement doesn't invalidate the significant DCG@10 gain observed for the other query type. Disabling the reranker everywhere would throw away a measured quality improvement that the evaluation clearly demonstrated for at least part of the traffic.
    • D. DCG@10 is a ranking-order-sensitive metric that is well suited to measuring exactly what reranking changes: the order of results within the top-k. Recall@k measures whether relevant items appear at all in the candidate set, which doesn't capture the reordering benefit reranking provides.

    Domain 3: Application Development

    Subdomain 3.2: Qualitatively assess responses to identify common issues such as quality and safety

    10.Which built-in MLflow GenAI judge requires a labeled ground-truth expected response in order to compute its score?

    1. A.Correctness, which compares the generated response against a labeled expected answer to score factual accuracy
    2. B.Safety, which is reference-free and flags harmful or toxic content without needing an expected answer
    3. C.RelevanceToQuery, which is reference-free and scores whether the response addresses the query's intent
    4. D.RetrievalGroundedness, which is reference-free and checks the response only against the retrieved context
    Show answer & explanation

    Correct answer: ACorrectness, which compares the generated response against a labeled expected answer to score factual accuracy

    • A. Correctness explicitly needs a labeled expected answer supplied as ground truth, against which it compares the generated response to score factual accuracy; without that label, the judge cannot run.
    • B. Safety is a reference-free judge that assesses whether content is harmful, offensive, or toxic based on the response alone, so it does not require any labeled expected answer.
    • C. RelevanceToQuery is reference-free and only needs the question and the response to judge whether the response addresses the query's intent.
    • D. RetrievalGroundedness is reference-free and checks whether the response is supported by the retrieved context, not by a labeled ground-truth answer.

    Subdomain 3.1: Select Langchain/similar tools for use in a Generative AI application.

    11.Which Python package provides purpose-built LangChain tool wrappers for Unity Catalog functions, letting a governed UC function be surfaced directly as a LangChain-compatible tool for an agent?

    1. A.databricks-langchain
    2. B.databricks-connect
    3. C.databricks-sdk
    4. D.databricks-feature-engineering
    Show answer & explanation

    Correct answer: Adatabricks-langchain

    • A. This package provides the LangChain integrations, including a toolkit that wraps Unity Catalog functions so they can be passed directly into a LangChain agent's tool list.
    • B. This package is a client for running Spark workloads against a remote Databricks cluster from a local environment; it has nothing to do with wrapping UC functions as LangChain tools.
    • C. This package is a general-purpose REST API client for managing Databricks workspace resources such as clusters and jobs, not a LangChain tool integration library.
    • D. This package is used for managing feature tables and feature lookups in the Feature Store, which is unrelated to exposing UC functions as LangChain tools.

    Subdomain 3.4: Augment a prompt with additional context from a user's input based on key fields, terms, and intents

    12.In the context of augmenting prompts based on a user's input, what is the primary purpose of extracting key fields, terms, and intents from that input?

    1. A.To determine which specific context and data should be retrieved and injected into the prompt so the LLM's response is grounded in relevant, accurate information
    2. B.To permanently store the user's raw message in a vector database for future model fine-tuning runs
    3. C.To validate that the user's message conforms to a fixed grammar before any processing can occur
    4. D.To compress the user's message into fewer tokens purely to reduce API billing costs
    Show answer & explanation

    Correct answer: ATo determine which specific context and data should be retrieved and injected into the prompt so the LLM's response is grounded in relevant, accurate information

    • A. Extracting key fields, terms, and intents lets the application identify exactly what additional context (records, documents, or data) is relevant, so it can be retrieved and injected into the prompt to ground the LLM's response, which is the core purpose of this augmentation step.
    • B. Storing raw messages in a vector database for fine-tuning is a separate model-training concern and is not the reason key fields and intents are extracted during prompt augmentation.
    • C. There is no requirement that user input conform to a fixed grammar; natural language inputs are inherently varied, and extraction is about identifying meaning, not enforcing syntax rules.
    • D. While shorter prompts can reduce cost, the purpose of extracting key fields and intents is to identify what context to retrieve, not primarily to minimize token usage.

    Subdomain 3.5: Create a prompt that adjusts an LLM's response from a baseline to a desired output

    13.An engineer is building a customer-facing chatbot for a retail app. The baseline prompt is just the user's question forwarded directly to the LLM, and the model sometimes answers in a formal, encyclopedic tone that does not match the brand's friendly, casual voice, and occasionally discusses unrelated product categories the store doesn't sell. What is the most direct prompt-engineering fix to consistently constrain both tone and topic scope?

    1. A.Prepend a system/instruction prompt that defines the assistant's persona, tone, and the specific product scope it is allowed to discuss, ahead of the user's question.
    2. B.Append the phrase 'be friendly' to the end of every user question before sending the combined text to the model.
    3. C.Retrain the underlying foundation model on a corpus of the brand's marketing copy so it internalizes the desired tone.
    4. D.Route every user question through a second LLM call that rewrites the first model's response into a more casual tone afterward.
    Show answer & explanation

    Correct answer: APrepend a system/instruction prompt that defines the assistant's persona, tone, and the specific product scope it is allowed to discuss, ahead of the user's question.

    • A. A system/instruction prompt that establishes persona, tone, and topic boundaries before the user turn is the standard mechanism for consistently shaping both style and scope across every interaction, since it applies as a stable constraint rather than a one-off hint.
    • B. Appending a short phrase only nudges tone weakly and inconsistently, and it does nothing to constrain which product categories the assistant is allowed to discuss, leaving the scope problem unaddressed.
    • C. Retraining or fine-tuning the base model is a heavyweight, costly approach reserved for cases where prompting cannot achieve the desired behavior; tone and scope framing here are directly solvable through prompt design without model retraining.
    • D. Adding a second rewriting pass could adjust tone after the fact but doubles latency and cost, and it still does not prevent the first model from discussing out-of-scope products, so the underlying scope issue persists.

    Subdomain 3.7: Select the best LLM based on the attributes of the application to be developed

    14.A legal team wants to summarize full contracts, some running 50 to 100 pages, in a single prompt without splitting the document into chunks. Which model attribute should be prioritized when selecting an LLM for this application?

    1. A.A large context window that can accept the entire contract in a single prompt, avoiding the need to split the document into chunks
    2. B.The highest reported benchmark score on coding tasks, since strong coding performance typically transfers to accurate long-document summarization
    3. C.The lowest cost per token, since summarization quality depends primarily on minimizing inference spend rather than input capacity
    4. D.Fine-tuning for real-time voice transcription, since transcription-focused training improves comprehension of long written contracts
    Show answer & explanation

    Correct answer: AA large context window that can accept the entire contract in a single prompt, avoiding the need to split the document into chunks

    • A large context window that can accept the entire contract in a single prompt, avoiding the need to split the document into chunks. Prioritizing a large context window is correct because the application's defining constraint is fitting long documents into one prompt, and an insufficient context window would force chunking and risk losing cross-section context in the contract.
    • The highest reported benchmark score on coding tasks, since strong coding performance typically transfers to accurate long-document summarization. This is incorrect because coding benchmarks measure a different capability and do not indicate whether a model can accept or reason over very long input documents.
    • The lowest cost per token, since summarization quality depends primarily on minimizing inference spend rather than input capacity. This is incorrect because a model with an inadequate context window is unusable for this task regardless of price, so input capacity is the binding constraint rather than cost.
    • Fine-tuning for real-time voice transcription, since transcription-focused training improves comprehension of long written contracts. This is incorrect because voice transcription tuning targets audio-to-text conversion and has no bearing on a model's ability to process long written legal text.

    Subdomain 3.8: Select an embedding model context length based on source documents, expected queries, and optimization strategy

    15.A social-media monitoring pipeline embeds individual posts averaging 50 tokens, with none exceeding 120 tokens, using the `databricks-gte-large-en` embedding model. Serving costs and query latency are higher than budgeted, and an audit finds no evidence that the model's extra context capacity is ever used. Which change best optimizes this pipeline without harming retrieval quality?

    1. A.Keep the `databricks-gte-large-en` embedding model, but batch multiple posts together into a single embedding request so the unused context capacity is filled and cost per post drops.
    2. B.Switch to the `databricks-bge-large-en` embedding model, since its 512-token context window comfortably covers every post while its smaller context window typically reduces per-request compute cost and latency.
    3. C.Keep the `databricks-gte-large-en` embedding model, but reduce the vector search index refresh frequency so fewer embedding calls are made per hour, lowering total serving cost.
    4. D.Switch to the `databricks-bge-large-en` embedding model, since it produces higher-dimensional embeddings than `databricks-gte-large-en` and therefore returns results faster for short text.
    Show answer & explanation

    Correct answer: BSwitch to the `databricks-bge-large-en` embedding model, since its 512-token context window comfortably covers every post while its smaller context window typically reduces per-request compute cost and latency.

    • A. Batching posts into one request packs several separate posts into a single call, but each post still needs to be tracked and embedded as distinct text for retrieval; this does not address the underlying issue of paying for an oversized context window that the workload never needs.
    • B. This is correct: since no post ever approaches even the smaller model's 512-token window, moving to this model still fully covers every post while typically lowering per-request compute cost and latency compared to a model provisioned for much longer text.
    • C. Reducing refresh frequency changes how often the index is updated with new content, not how much compute each embedding call consumes, so the pipeline keeps paying for context capacity the short posts never use.
    • D. This claim is factually incorrect: `databricks-bge-large-en` and `databricks-gte-large-en` both produce 1024-dimension embeddings, so the smaller model's benefit here comes from its smaller context window rather than any dimensionality difference.

    Subdomain 3.9: Select a model from a model hub or marketplace for a task based on model metadata/model cards

    16.A team is choosing between two candidate LLMs listed on Databricks Marketplace for a latency-sensitive, high-volume customer support chatbot. Model X has a higher MMLU benchmark score but a much larger parameter count and higher measured serving latency; Model Y has a slightly lower benchmark score, a smaller parameter count, and meets the application's response-time requirement. Based on the model cards, which model should the team select?

    1. A.Model X, because a higher MMLU benchmark score always outweighs serving latency in production systems
    2. B.Model Y, because it satisfies the latency requirement while offering benchmark quality close to Model X's score
    3. C.Model X, because larger parameter counts guarantee better real-world accuracy regardless of the benchmark used
    4. D.Model Y, but only after fine-tuning it further to exactly match Model X's total parameter count and architecture
    Show answer & explanation

    Correct answer: BModel Y, because it satisfies the latency requirement while offering benchmark quality close to Model X's score

    • A. Treating benchmark score as the only decision factor ignores the application's stated latency requirement; a chatbot that cannot respond within the required time degrades the user experience regardless of its raw accuracy score.
    • B. Selecting the model that meets the latency requirement while remaining close in benchmark quality is correct because model selection should weigh application attributes like response-time constraints alongside evaluation metrics, not optimize for a single metric in isolation.
    • C. A larger parameter count does not guarantee better real-world accuracy; benchmark scores already capture measured quality, and here the smaller model's score is only slightly lower while its latency profile fits the use case.
    • D. Fine-tuning a smaller model specifically to match a larger model's parameter count and architecture is not a real or necessary step, since parameter count is an inherent architectural property rather than something fine-tuning changes, and doing so would defeat the latency advantage.

    Subdomain 3.10: Select the best model for a given task based on common metrics generated in experiments

    17.While selecting a foundation model for a support-ticket classification task, an engineer notices that Model X has higher published scores on general-purpose leaderboards than Model Y. However, an MLflow evaluation run on the company's own labeled ticket dataset shows Model Y outperforming Model X for this specific task. Which model should be selected, and why?

    1. A.Select Model X, because published leaderboard benchmarks are always more statistically reliable than results from a single internal evaluation dataset
    2. B.Select Model Y, because task-specific evaluation results from the company's own labeled dataset are more representative of production performance than general-purpose benchmark scores
    3. C.Select Model X, because model card metadata takes precedence over MLflow experiment metrics whenever the two sources disagree on model quality
    4. D.Select whichever model has the lower listed parameter count, because smaller models are assumed to generalize better to task-specific applications than the leaderboard results suggest
    Show answer & explanation

    Correct answer: BSelect Model Y, because task-specific evaluation results from the company's own labeled dataset are more representative of production performance than general-purpose benchmark scores

    • A. General-purpose leaderboard scores measure performance on broad, generic tasks and don't account for the specific distribution of the company's support tickets, so they aren't inherently more reliable than a task-specific evaluation.
    • B. Correct. An evaluation run on data that reflects the actual application, such as the company's own labeled ticket dataset, more accurately predicts production performance for that task than a general-purpose leaderboard score.
    • C. Model card metadata provides general reference information, but it doesn't override task-specific experiment results when the two disagree; the internal evaluation is more directly relevant to this application.
    • D. Parameter count alone doesn't determine how well a model generalizes to a specific classification task, and this option ignores the actual evaluation metrics that were generated for the two candidates.

    Subdomain 3.11: Utilize MLflow and Agent Framework for developing agentic systems

    18.A developer is prototyping a retrieval tool locally in a notebook before deploying an agent that queries a Databricks-managed vector search index built with Databricks-managed embeddings. They want a LangChain-compatible tool object they can pass directly to their agent with minimal setup. Which option fits this stage of development?

    1. A.Install `databricks-langchain` and instantiate `VectorSearchRetrieverTool` with the `index_name` and `tool_description` parameters.
    2. B.Install `databricks-openai` and instantiate `DatabricksEmbeddings` directly as the callable tool passed to the agent's tool list.
    3. C.Deploy the index behind the managed AI Search MCP server first and connect only through `DatabricksMCPClient` for all local prototyping.
    4. D.Create a Unity Catalog connection object pointing at the vector index and define a retriever function that calls the connection directly.
    Show answer & explanation

    Correct answer: AInstall `databricks-langchain` and instantiate `VectorSearchRetrieverTool` with the `index_name` and `tool_description` parameters.

    • A. This is correct because this class from the Databricks AI Bridge package is purpose-built to wrap a Databricks-managed vector index as a ready-to-use LangChain tool with minimal setup for local prototyping.
    • B. This is incorrect because an embeddings class produces vector representations of text; it is not itself a retriever tool and cannot be passed to an agent as a callable tool.
    • C. This is incorrect because standing up a managed MCP server is the recommended path for production-grade, governed access, but it adds infrastructure overhead unnecessary for quick local prototyping against an already Databricks-managed index.
    • D. This is incorrect because the Unity Catalog connection pattern is intended for indexes or vector stores hosted outside Databricks, not for an index that already uses Databricks-managed embeddings and hosting.

    Subdomain 3.13: Enable multi-agent systems to leverage Genie Spaces or conversational API to retrieve data

    19.An agent has already sent a message to a Genie space and received a `message_id`. It now needs to obtain the tabular result of the query Genie generated so it can pass the rows back to the orchestrating agent. What is the correct sequence to follow?

    1. A.Poll `GET .../messages/{message_id}` until the status is conclusive (such as completed), then call `GET .../messages/{message_id}/query-result/{attachment_id}` to fetch the rows.
    2. B.Call `GET .../messages/{message_id}/query-result/{attachment_id}` immediately, since the query result is always available synchronously as soon as the message is created.
    3. C.Call `DELETE .../conversations/{conversation_id}` to force Genie to finalize the query, then re-issue `start-conversation` to retrieve the finished rows.
    4. D.Poll `POST /api/2.0/genie/spaces/{space_id}/start-conversation` repeatedly with the same message text until the response includes a populated result set.
    Show answer & explanation

    Correct answer: APoll `GET .../messages/{message_id}` until the status is conclusive (such as completed), then call `GET .../messages/{message_id}/query-result/{attachment_id}` to fetch the rows.

    • A. This is correct because message processing is asynchronous: the agent must poll the message endpoint until a conclusive status is reached before the generated SQL's results become retrievable via the query-result endpoint keyed by attachment ID.
    • B. This is incorrect because SQL generation and execution take time and are not guaranteed to finish synchronously; fetching the query result before processing completes will not reliably return the finished rows.
    • C. This is incorrect because deleting a conversation removes it rather than finalizing it, and starting a brand-new conversation discards the in-progress question instead of retrieving its results.
    • D. This is incorrect because repeatedly calling start-conversation creates a new conversation thread each time rather than checking the status of the message that was already submitted.

    Subdomain 3.14: Select the best model for a given task based on common metrics generated in experiments

    20.A team is deploying a customer-facing chat agent and has three candidate LLMs, each logged as an MLflow run with a quality judge score, average token latency, and per-1M-token cost recorded as run metrics. The product requirement is sub-second responses at the lowest cost while still clearing a minimum quality bar. Which approach best uses the logged experiment metrics to select the deployment model?

    1. A.Filter the runs to those meeting the minimum quality score, then among the remaining runs choose the one with the lowest combined latency and cost.
    2. B.Select the model with the highest quality judge score regardless of its logged latency or cost metrics, since customer satisfaction outweighs the other run values.
    3. C.Select the model with the lowest per-1M-token cost first, then verify its quality judge score after deployment using production inference tables.
    4. D.Select the model with the lowest average token latency, since sub-second response time is stated as a requirement and cost can be optimized with provisioned throughput later.
    Show answer & explanation

    Correct answer: AFilter the runs to those meeting the minimum quality score, then among the remaining runs choose the one with the lowest combined latency and cost.

    • A. Filtering out runs that fail the quality threshold before comparing latency and cost across the remaining candidates directly uses all three logged metrics to satisfy the quality bar, latency target, and cost goal together.
    • B. Ignoring the logged latency and cost metrics risks selecting a model that violates the sub-second response requirement or exceeds the cost target, even though it scored well on quality.
    • C. Choosing purely on cost and deferring quality verification to after deployment abandons the experiment metrics that were already collected and risks shipping a model that fails the minimum quality bar.
    • D. Optimizing only for latency ignores the logged quality and cost metrics, so the selected model could fail the minimum quality bar or exceed the target cost despite meeting the latency goal.

    Domain 4: Assembling and Deploying Applications

    Subdomain 4.3: Code a simple chain according to requirements

    21.A generative AI engineer is assembling a simple chain in LangChain on Databricks that takes a user question, applies a `ChatPromptTemplate`, sends it to a `ChatDatabricks` LLM, and parses the output with a `StrOutputParser`. Which code correctly composes these three components into a single runnable chain using LangChain Expression Language?

    1. A.Chain the components with the pipe operator: `prompt | llm | output_parser`, composing them into one Runnable in sequence.
    2. B.Wrap the components in `LLMChain(prompt, llm, output_parser)`, passing all three arguments positionally to build the chain.
    3. C.Wrap the components in `SequentialChain(chains=[prompt, llm, output_parser])`, listing each step in the chains parameter.
    4. D.Call `RunnableSequence(steps=[prompt, llm, output_parser])`, passing each component as an ordered list of steps.
    Show answer & explanation

    Correct answer: AChain the components with the pipe operator: `prompt | llm | output_parser`, composing them into one Runnable in sequence.

    • A. Correct: the pipe operator is LangChain Expression Language's standard syntax for composing Runnables, so linking the prompt template, chat model, and output parser this way builds a single sequential chain.
    • B. Incorrect: `LLMChain` is the legacy chain class and is not constructed by passing a prompt, chat model, and parser as three positional arguments in this way.
    • C. Incorrect: `SequentialChain` composes multiple named sub-chains with declared input/output keys for multi-step workflows, it is not how a single prompt-LLM-parser pipeline built from these three objects is expressed.
    • D. Incorrect: `RunnableSequence` is not instantiated with a `steps` keyword this way; the pipe operator is the documented syntax that builds a `RunnableSequence` under the hood.

    Subdomain 4.2: Control access to resources from model serving endpoints

    22.A team is deploying an agent whose Unity Catalog table queries must respect each individual end user's own row- and column-level permissions rather than a single shared identity's permissions. Which configuration achieves this on a Model Serving endpoint?

    1. A.Enable on-behalf-of-user authentication, build the client with `ModelServingUserCredentials` inside `predict`, and declare `api_scopes` in an `AuthPolicy` at logging time.
    2. B.Declare the tables as resource dependencies with mlflow.models.resources so the system-generated service principal gets read-only access shared by every caller.
    3. C.Store one service principal's personal access token as a Databricks secret and reference it via `{{secrets/scope/key}}` so all requests authenticate identically.
    4. D.Grant the workspace's default Unity Catalog metastore admin role to the endpoint so it bypasses table-level access checks for every incoming request.
    Show answer & explanation

    Correct answer: AEnable on-behalf-of-user authentication, build the client with `ModelServingUserCredentials` inside `predict`, and declare `api_scopes` in an `AuthPolicy` at logging time.

    • A. On-behalf-of-user authentication is the feature designed for this scenario: initializing `ModelServingUserCredentials` inside the prediction function (since user identity is only known at request time) and declaring the needed API scopes lets each request enforce that specific user's own Unity Catalog permissions.
    • B. Automatic authentication passthrough with a declared resource grants the same service-principal-level access to every caller of the endpoint, so it cannot differentiate row- or column-level permissions between individual end users.
    • C. Referencing a single service principal's token means every request authenticates as that one identity, which enforces uniform access rather than each end user's individual row- and column-level entitlements.
    • D. Granting a broad metastore admin role bypasses access checks entirely rather than applying each user's own permissions, which is the opposite of enforcing per-user row- and column-level governance.

    Subdomain 4.6: Create and query a Vector Search index

    23.An agent's retrieval tool queries a vector search index and needs to combine semantic similarity with exact keyword matching to improve recall for exact product names, while also restricting results to documents where `category` equals `electronics`. Which combination of query call parameters achieves this?

    1. A.Set `query_type="HYBRID"` and pass `filters={"category": "electronics"}` in the `similarity_search` call.
    2. B.Set `query_type="ANN"` and pass `filters={"category": "electronics"}`, since ANN already blends keyword and vector scoring.
    3. C.Set `query_type="HYBRID"` and pass `columns=["category"]` to restrict the search to documents in that category.
    4. D.Set `query_type="FULL_TEXT"` and pass `filters={"category": "electronics"}`, since full-text indexes always blend semantic scoring.
    Show answer & explanation

    Correct answer: ASet `query_type="HYBRID"` and pass `filters={"category": "electronics"}` in the `similarity_search` call.

    • A. HYBRID query type combines vector similarity with keyword matching, and the `filters` argument restricts results to matching metadata, together satisfying both requirements.
    • B. ANN performs pure approximate nearest-neighbor vector search only; it does not blend in keyword scoring, so it would not improve recall for exact product-name matches.
    • C. The `columns` parameter only controls which fields are returned in the results; it does not restrict which documents are searched, so category filtering would not actually apply.
    • D. A full-text query type performs keyword-based search and does not automatically blend in semantic vector scoring, so it would not provide the combined similarity behavior needed.

    Subdomain 4.9: Identify batch inference workloads and apply ai_query() appropriately

    24.A batch summarization job calls `ai_query()` against a foundation model endpoint for every article in a table, but the generated summaries are consistently too long and vary unpredictably in style between runs. The team wants shorter, more consistent output without changing the prompt text. Which addition to the `ai_query()` call addresses this?

    1. A.Pass `modelParameters => named_struct('max_tokens', 100, 'temperature', 0.2)` to cap the generation length and reduce output variability.
    2. B.Pass `returnType => 'STRING'` explicitly so the engine truncates any response longer than the inferred custom endpoint schema allows.
    3. C.Wrap the endpoint name in a `named_struct()` so the request is treated as a structured custom-model call with a fixed output length.
    4. D.Set `files => content` on the call so the request is processed as multimodal input with a smaller default response budget.
    Show answer & explanation

    Correct answer: APass `modelParameters => named_struct('max_tokens', 100, 'temperature', 0.2)` to cap the generation length and reduce output variability.

    • A. Correct: `modelParameters` carries chat/completion controls like `max_tokens` and `temperature` straight through to the foundation model endpoint, so capping `max_tokens` bounds summary length and lowering `temperature` reduces run-to-run variability.
    • B. `returnType` describes the expected schema of a custom endpoint's response for parsing purposes; it is not a truncation mechanism and has no effect on how much text a foundation chat model generates.
    • C. The endpoint argument is a plain string identifying which serving endpoint to call; wrapping it in a struct does not change generation length and would break the call rather than configure output size.
    • D. `files` is used to attach image content for multimodal requests; it is unrelated to text summarization and does not influence response length or determinism for a text-only prompt.

    Subdomain 4.8: Explain the key concepts and components of Mosaic AI Vector Search

    25.When evaluating AI Search retrieval quality, which metric does Databricks recommend as the primary metric because it weighs highly relevant results more heavily and accounts for their rank position in the result list?

    1. A.Recall@10
    2. B.Precision@10
    3. C.DCG@10
    4. D.MRR
    Show answer & explanation

    Correct answer: CDCG@10

    • A. Recall@10 measures the fraction of all relevant documents that were retrieved in the top 10 results, but it does not weight results by degree of relevance or by their position within that top 10. It is not the metric Databricks highlights as primary for this reason.
    • B. Precision@10 measures the fraction of the top 10 retrieved results that are relevant, treating all relevance levels and positions within the top 10 equally. It does not capture the graded, position-weighted signal that the recommended metric provides.
    • C. Discounted Cumulative Gain at 10 sums graded relevance scores while discounting results that appear lower in the ranking, so highly relevant documents near the top contribute more than marginally relevant ones further down. This combination of graded relevance and rank-position weighting is why Databricks recommends it as the primary retrieval quality metric.
    • D. Mean Reciprocal Rank only considers the position of the first relevant result and ignores the graded relevance or ranking of any other results in the list. It does not account for multiple relevant documents at varying relevance levels the way the recommended metric does.

    Subdomain 4.10: Configure vector search for a particular solution based on number of embeddings, update frequency, latency, and cost requirements.

    26.A customer-facing search feature must guarantee low query latency at a consistently high query volume, and the business has approved additional spend to meet this SLA. Which configuration choice best supports this requirement?

    1. A.Set the `target_qps` parameter on the index to reserve additional serving capacity, accepting that the endpoint is billed for that reserved capacity even during idle periods.
    2. B.Enable scale-to-zero on the serving endpoint so capacity is only provisioned when queries arrive, keeping costs low during idle periods.
    3. C.Switch the index to Triggered sync mode so queries are processed in scheduled batches instead of continuously reserving compute.
    4. D.Leave `target_qps` unset and rely on default best-effort scaling, since Databricks automatically reserves capacity once high traffic is detected.
    Show answer & explanation

    Correct answer: ASet the `target_qps` parameter on the index to reserve additional serving capacity, accepting that the endpoint is billed for that reserved capacity even during idle periods.

    • A. Setting `target_qps` reserves dedicated serving capacity to meet the requested query throughput, and that reserved capacity is billed regardless of actual traffic, which matches a business that has approved extra spend for a latency guarantee.
    • B. Scale-to-zero shuts down capacity during idle periods, which introduces a warm-up delay of a couple of minutes when traffic resumes, directly undermining a requirement for guaranteed low latency at high volume.
    • C. Sync mode governs how the index ingests updates from the source table, not how query traffic is served, so changing it has no effect on query latency or throughput guarantees.
    • D. Without `target_qps` set, scaling is best-effort and not guaranteed, so the endpoint may not reserve enough capacity to hold latency steady under sustained high query volume.

    Subdomain 4.11: Configure a persistent datastore to store and retrieve intermediate memory or structured information.

    27.What is Databricks Lakebase, as used for self-managed agent memory?

    1. A.A managed Unity Catalog securable object that stores memory entries under a specific scope and path.
    2. B.A vector index service that stores and retrieves embeddings for semantic similarity search.
    3. C.A batch scheduling service that triggers periodic Delta Lake table maintenance jobs.
    4. D.A fully managed, serverless Postgres OLTP database that agents use as a durable memory store.
    Show answer & explanation

    Correct answer: DA fully managed, serverless Postgres OLTP database that agents use as a durable memory store.

    • A. A Unity Catalog securable memory store with scoped, path-organized entries describes managed agent memory, which is a separate, Databricks-governed approach rather than what Lakebase itself is.
    • B. Storing and retrieving embeddings for similarity search describes a vector search index, not Lakebase, which is a relational Postgres database rather than an embedding index.
    • C. Lakebase is an operational Postgres database for transactional workloads, not a scheduling service, and it has no role in triggering Delta Lake table maintenance jobs.
    • D. Lakebase is Databricks' fully managed, serverless Postgres OLTP database, and self-managed agent memory uses it as a durable store for short-term and long-term agent state such as checkpoints and conversation history.

    Subdomain 4.13: Integrate managed, external, and custom MCP servers based on a given application requirements

    28.A team is building an agent that must run governed Unity Catalog SQL functions as predefined tools, without standing up any new infrastructure. The workspace already has the functions registered in a catalog and schema, and the team wants Unity Catalog permissions to govern which users can invoke each function. Which approach best satisfies this requirement?

    1. A.Connect the agent to the managed Unity Catalog Functions MCP server at `/api/2.0/mcp/functions/{catalog}/{schema}`, letting Unity Catalog permissions govern access.
    2. B.Deploy a custom MCP server as a Databricks App that wraps each Unity Catalog function in a `@mcp.tool()` decorator and exposes it over the app's `/mcp` path.
    3. C.Register the Unity Catalog schema as an external MCP server through a Unity Catalog connection with managed OAuth so the agent reaches it as a third-party service.
    4. D.Have the agent call the functions directly through the SQL warehouse REST API and enforce access with a workspace-level access token instead of Unity Catalog grants.
    Show answer & explanation

    Correct answer: AConnect the agent to the managed Unity Catalog Functions MCP server at `/api/2.0/mcp/functions/{catalog}/{schema}`, letting Unity Catalog permissions govern access.

    • A. This is correct: Databricks hosts a managed Unity Catalog Functions MCP server that exposes registered SQL/Python functions as predefined tools at a workspace endpoint, with Unity Catalog enforcing which principals can call which function, so no additional hosting is required.
    • B. Building a custom MCP server on Databricks Apps is unnecessary extra work here since the functions are already governed Unity Catalog objects that the managed Unity Catalog Functions server can expose directly without any app code or deployment.
    • C. External MCP registration through Unity Catalog connections is meant for servers hosted outside Databricks; functions that already live in a Unity Catalog schema inside the workspace are served by the managed server, not treated as an external third-party connection.
    • D. Calling the SQL warehouse REST API directly bypasses the MCP tool-calling contract the agent framework expects and substitutes a workspace token for Unity Catalog's fine-grained function-level grants, weakening governance rather than preserving it.

    Domain 5: Governance

    Subdomain 5.2: Select guardrail techniques to protect against malicious user inputs to a Gen AI application

    29.An engineering team needs to block requests that mention specific unreleased internal product codenames, a policy unique to their organization with no equivalent among the built-in PII, unsafe content, jailbreak, or hallucination templates. Which guardrail option lets them define this policy?

    1. A.A custom guardrail with a user-defined prompt sent to an evaluator endpoint, configured to block requests matching the organization-specific policy.
    2. B.The unsafe content guardrail, retuned to treat internal product codenames as an additional category of hate speech or harassment.
    3. C.The jailbreak protection guardrail, extended to treat any mention of a codename as a role-play exploit attempt requiring a block.
    4. D.The hallucination detection guardrail, applied to flag any codename mention as a fabricated or non-existent product reference.
    Show answer & explanation

    Correct answer: AA custom guardrail with a user-defined prompt sent to an evaluator endpoint, configured to block requests matching the organization-specific policy.

    • A. Custom guardrails accept a user-defined prompt, up to 5000 characters, evaluated against a compatible endpoint, which lets a team codify an organization-specific policy like blocking mentions of internal codenames that no built-in template covers.
    • B. The unsafe content category is fixed to detect things like hate speech, violence, and self-harm; it has no mechanism to be retuned toward an unrelated, organization-specific list of confidential codenames.
    • C. Jailbreak protection is fixed to detect instruction overrides, obfuscated payloads, and role-play exploits, not arbitrary organization-specific terms, so it cannot be repurposed to flag product codenames.
    • D. Hallucination detection is fixed to catch fabricated facts and invented citations in model output; a real but confidential codename mentioned in a request is not a fabrication, so this guardrail would not apply.

    Subdomain 5.1: Use masking techniques as guard rails to meet a performance objective

    30.A RAG application's retrieval layer queries a claims table where analysts in the `us-team` group should see only rows for the `US` region, while the claim amount values themselves must remain fully visible to every analyst. Which governance control satisfies this requirement?

    1. A.Attach a row filter function that excludes rows where the region does not match the querying user's team, leaving every column value unmasked.
    2. B.Attach a column mask to the `region` column that returns a placeholder string for users outside the `us-team` group, leaving all rows visible.
    3. C.Attach a column mask to the `claim_amount` column that zeroes out the value for users outside the `us-team` group, leaving all rows visible.
    4. D.Attach a row filter function that excludes rows where the claim amount exceeds a threshold for users outside the `us-team` group.
    Show answer & explanation

    Correct answer: AAttach a row filter function that excludes rows where the region does not match the querying user's team, leaving every column value unmasked.

    • A. A row filter is designed to exclude entire rows based on a condition like matching region to team, which restricts which claims an analyst can see while leaving the claim amount column itself fully visible in the rows that remain.
    • B. Masking the region column would hide which region a visible row belongs to rather than restrict which rows are returned, so analysts would still see every row's claim amount regardless of region, failing the row-level requirement.
    • C. Masking the claim_amount column directly contradicts the requirement that claim amounts remain fully visible to every analyst, and it does nothing to restrict rows by region.
    • D. Filtering rows by claim amount threshold restricts access based on the wrong attribute, since the requirement is to restrict rows by region, not by the size of the claim amount.

    Subdomain 5.3: Use legal/licensing requirements for data sources to avoid legal risk

    31.An enterprise wants its internal RAG assistant to retrieve full-text passages from a vendor's copyrighted technical manuals that the company already licenses for employee use. What is the correct way to incorporate this content into the RAG knowledge base?

    1. A.Confirm that the existing vendor license agreement permits internal retrieval-augmented use of the manuals, then restrict retrieval access to the employees covered by that license using Unity Catalog permissions
    2. B.Assume internal-only use is automatically exempt from the vendor's copyright, since the manuals never leave the company's own environment
    3. C.Add a footer crediting the vendor to each retrieved passage, since attribution alone satisfies copyright obligations for licensed manuals
    4. D.Redact any personally identifiable information from the manuals before ingestion, since removing PII resolves the underlying copyright status of the content
    Show answer & explanation

    Correct answer: AConfirm that the existing vendor license agreement permits internal retrieval-augmented use of the manuals, then restrict retrieval access to the employees covered by that license using Unity Catalog permissions

    • A. Existing vendor licenses often specify permitted uses and user populations, so the compliant approach is to confirm the license actually covers retrieval-augmented use and then technically enforce that scope with Unity Catalog access controls. This ties the legal permission to an enforceable technical boundary.
    • B. Staying within company infrastructure does not exempt copyrighted material from the terms of the license under which it was obtained; internal use can still exceed what the vendor agreement authorizes. The license scope, not the deployment boundary, determines what is permitted.
    • C. Attribution can be one license requirement among several, but it does not by itself satisfy broader copyright terms such as limits on redistribution, number of users, or permitted use cases. Crediting the vendor is not a substitute for confirming the license actually allows this use.
    • D. PII redaction addresses privacy concerns about individuals' personal data, which is unrelated to whether the company holds sufficient copyright permission to reuse the vendor's manuals. Removing PII has no effect on the underlying license status of the copyrighted text.

    Subdomain 5.4: Recommend an alternative for problematic text mitigation in a data source feeding a GenAI application

    32.A support team builds a RAG chatbot whose knowledge base is populated from scraped public complaint-forum posts. Reviewers notice that some retrieved passages contain profanity and hateful language, which occasionally surfaces in the chatbot's responses. What should the team do to address this at the data source level?

    1. A.Run the scraped posts through a toxicity/profanity classifier during data preparation and remove or quarantine flagged passages before they are indexed
    2. B.Lower the temperature parameter on the serving endpoint so the model paraphrases retrieved text more conservatively
    3. C.Instruct the model with a system prompt to politely decline whenever a retrieved passage contains offensive language
    4. D.Increase the number of retrieved chunks per query so offensive passages are diluted among more neutral context
    Show answer & explanation

    Correct answer: ARun the scraped posts through a toxicity/profanity classifier during data preparation and remove or quarantine flagged passages before they are indexed

    • A. Filtering or quarantining problematic passages during data preparation stops offensive content from ever entering the index, which is the most reliable way to prevent it from being retrieved and surfaced in responses.
    • B. Temperature controls the randomness of token sampling during generation; it does not screen or remove offensive content from the underlying source documents, so the problematic text remains retrievable.
    • C. A system prompt instruction depends on the model reliably recognizing and reacting to offensive retrieved content every time, which is an unreliable inference-time patch rather than a fix to the source data itself.
    • D. Retrieving more chunks does not remove the offensive passages from the corpus; it can actually increase the odds that at least one problematic passage is included in the context window.

    Domain 6: Evaluation and Monitoring

    Subdomain 6.1: Select an LLM choice (size and architecture) based on a set of quantitative evaluation metrics

    33.A retrieval-augmented question-answering system is being evaluated with two candidate LLMs on the same held-out test set of question-answer pairs. Model A scores 88% on an answer-correctness metric with an average response time of 1.2 seconds. Model B scores 89% on the same metric with an average response time of 4.5 seconds. The application has a strict service-level requirement that responses must return in under 2 seconds. Based on this evaluation data, what is the correct model selection decision?

    1. A.Model A should be selected, because Model B's 4.5-second response time violates the 2-second latency requirement despite its marginally higher correctness score
    2. B.Model B should be selected, because a higher score on the correctness metric should always outweigh latency considerations in evaluation-based model selection
    3. C.Model A should be selected, because any accuracy difference smaller than five percentage points should be treated as statistically meaningless and ignored entirely
    4. D.Model B should be selected, because latency measurements taken during evaluation do not reflect the latency the model will exhibit in production
    Show answer & explanation

    Correct answer: AModel A should be selected, because Model B's 4.5-second response time violates the 2-second latency requirement despite its marginally higher correctness score

    • A. This is correct: the application defines a hard latency requirement under 2 seconds, and Model B's measured 4.5-second average response time fails that requirement, so Model A is the only candidate that satisfies both the accuracy bar and the latency constraint.
    • B. This is incorrect because model selection must weigh all relevant quantitative requirements together; a correctness edge does not override a hard latency service-level requirement that the application has defined.
    • C. This is incorrect because there is no such blanket statistical rule; the scenario asks for a decision based on the stated latency requirement, not on discounting the accuracy gap outright.
    • D. This is incorrect because evaluation-time latency measurements are a standard and relevant proxy for expected production latency, and dismissing them contradicts the purpose of measuring latency during evaluation.

    Subdomain 6.5: Use Databricks features to control LLM costs

    34.A platform team supports five separate agent applications, each tagged with its own cost-center identifier, all routed through a shared Unity Catalog AI Gateway endpoint. Finance wants an email alert the moment any one cost center's monthly spend on that endpoint crosses $2,000, and they want the option to automatically block further requests from that cost center once the threshold is hit. Which Databricks feature should the team configure?

    1. A.A budget scoped to the cost-center tags with a defined threshold and usage blocking enabled for the AI Gateway endpoint.
    2. B.A rate limit on the endpoint expressed in tokens per minute, applied uniformly across all five cost-center tags.
    3. C.An inference table capturing every request and response payload for the endpoint, reviewed weekly by finance.
    4. D.A Unity Catalog access control list restricting which cost centers may query the shared endpoint at all.
    Show answer & explanation

    Correct answer: AA budget scoped to the cost-center tags with a defined threshold and usage blocking enabled for the AI Gateway endpoint.

    • A. Databricks budgets can be scoped to custom tags such as cost-center identifiers, tracking spend against a defined monthly threshold and sending near real-time alerts for Unity AI Gateway usage, with an optional usage-blocking setting that halts further requests once the threshold is reached, matching both the alerting and blocking requirements.
    • B. A tokens-per-minute rate limit caps request throughput rather than tracking dollar spend against a threshold, and applying one limit uniformly across all cost centers would not distinguish which specific cost center crossed $2,000, so it does not satisfy the tag-based spend alerting requirement.
    • C. An inference table logs request and response payloads for auditing, evaluation, and monitoring quality, but it has no built-in mechanism to compute dollar spend per tag, trigger a threshold alert, or block further calls.
    • D. An access control list can permit or deny which principals may query an endpoint, but it does not track cumulative spend or generate a threshold-crossing alert; it would only enforce an all-or-nothing access decision, not a cost-based one.

    Subdomain 6.9: Use Databricks custom Scorers for evaluating agents and LLMs

    35.A team writes a custom scorer for a support agent and wants stakeholders reviewing evaluation runs to see not just a numeric score but also a short written reason for that score, attached to each trace. What should the `@scorer`-decorated function return to achieve this?

    1. A.Return a `Feedback` object carrying a numeric `value` plus a `rationale` string explaining why that score was assigned
    2. B.Return a plain `float` between 0 and 1, since numeric scores are the only values MLflow can attach to a trace
    3. C.Return a `list` of raw floats, one per stakeholder, since only lists let a scorer report more than one value
    4. D.Return a `str` containing both the score and reasoning combined into one sentence, since strings support free text
    Show answer & explanation

    Correct answer: AReturn a `Feedback` object carrying a numeric `value` plus a `rationale` string explaining why that score was assigned

    • A. A `Feedback` object is built for exactly this case: it pairs a `value` (the score) with a `rationale` (the explanation), giving stakeholders both the number and the reasoning behind it in a structured way attached to the trace.
    • B. A plain `float` is a valid scorer return type, but it carries only the number with no place to attach an explanation, so it cannot satisfy the requirement for a visible written rationale.
    • C. A list return is meant for reporting several distinct named metrics from one scorer, not for pairing a single score with an explanation; it does not provide a structured rationale field either.
    • D. A `str` can hold text, but jamming a score and reasoning into one unstructured sentence loses the distinct, queryable `value` and `rationale` fields that a `Feedback` object provides.

    Want the full experience?

    These are just samples. Practice the full Databricks Certified Generative AI Engineer Associate question bank in quiz mode — free, no signup, with domain practice and exam simulation.