What you will be able to do
- Describe the Declarative Automation Bundles pipeline that validates, deploys and runs an agent
- Choose how a Vector Search index is kept current: TRIGGERED sync, CONTINUOUS sync, or direct upserts
- Layer tests from unit tests through bundle validation, local runs, load tests and a post-deploy smoke test
1.The agent's CI/CD pipeline with Declarative Automation Bundles
Databricks recommends Declarative Automation Bundles (formerly Databricks Asset Bundles) for CI/CD. A bundle describes resources as source files: code, jobs, apps and their configuration. You keep the bundle in Git and deploy it as one unit from GitHub Actions, Azure DevOps or Jenkins. The general guidance applies directly to agents. Keep everything under version control. Pass environment-specific settings as parameters instead of hard-coding them. Use separate workspaces for development, staging and production. Automate rollback when a deployment fails. Authenticate with workload identity federation, which removes the need for Databricks secrets.
For an agent on Databricks Apps, a configured pipeline deploys and restarts the agent every time a change merges to main. Several templates in databricks/app-templates already include a .github/workflows/deploy.yml. Your job is to make sure its databricks bundle run step uses the resource key from your databricks.yml. Within the pipeline, databricks bundle validate checks the configuration, databricks bundle deploy pushes it to a target, and databricks bundle run starts the resource.
Checkpoint 1 of 4· Put it in order
Put the core bundle steps of an agent deploy pipeline in order
- 1.databricks bundle run <bundle-key> --target prod
- 2.databricks bundle deploy --target prod
- 3.databricks bundle validate
- 4.Smoke test: POST a canary request to /invocations
Validation catches configuration mistakes before anything is deployed. Deploy pushes the bundle and run starts the app. Because run can return before the agent has finished booting, the smoke test comes last.
“Run databricks bundle validate before you deploy to catch YAML configuration issues early.”Source: docs.databricks.com
Agent templates ship with MLFLOW_EXPERIMENT_ID empty in databricks.yml. The quickstart script fills it in locally, but a fresh CI runner does not. Commit the numeric experiment_id. If you deploy to several workspaces, declare one experiment per targets.<env> block or use a bundle variable, because the experiment is scoped to a workspace.
2.Updating the Vector Search index in the pipeline
The retriever is part of the agent, so its index should be defined and updated in the same pipeline. The bundle resources reference includes a specification for a Delta Sync Index. It names a source_table, the embedding source or embedding vector columns, optional columns_to_sync, and a pipeline_type. The primary key and embedding columns are always synced. Defining the index this way puts its configuration in Git and deploys it with the rest of the app.
pipeline_type decides how the index gets updated. With CONTINUOUS, the index processes new source data as it arrives and nothing else is needed. With TRIGGERED, the pipeline must start a sync after the source table changes, for example as a CI or job step using the CLI. A Direct Vector Access index has no source table to sync from, so you write data to it with upserts.
| Index configuration | How new data reaches the index | Pipeline action |
|---|---|---|
| Delta Sync, pipeline_type TRIGGERED | Refreshes once per triggered update | databricks vector-search-indexes sync-index INDEX_NAME |
| Delta Sync, pipeline_type CONTINUOUS | Processes new source-table data as it arrives | None beyond updating source_table |
| Direct Vector Access | Data is upserted directly | databricks vector-search-indexes upsert-data-vector-index INDEX_NAME INPUTS_JSON |
databricks vector-search-indexes sync-index my-delta-sync-indexCheckpoint 2 of 4· Check yourself
A CI job adds a sync-index step for an index that the agent writes to with upsert-data-vector-index. What happens?
An index that receives upserts is a Direct Vector Access index. sync-index works only on Delta Sync indexes, which read from a source table.
“Name of the vector index to synchronize. Must be a Delta Sync Index.”Source: docs.databricks.com
Checkpoint 3 of 4· Exam question
A team stores its agent's system prompt in the MLflow Prompt Registry and loads it in production code using the URI `prompts:/catalog.schema.answer_prompt@production`. After validating a new prompt version in staging, they need to promote it to production as part of their CI/CD pipeline without changing or redeploying the application code. Which action accomplishes this?
Correct answer: A — Call `set_prompt_alias()` to repoint the `production` alias to the newly validated prompt version number
- A. Aliases are mutable pointers to specific immutable prompt versions, so repointing the production alias to the newly tested version promotes it while every consumer that loads via the `@production` alias picks up the change automatically. No code redeploy is needed because the URI itself never changes.
- B. Registering a duplicate version and then hardcoding its version number in application code requires a code change and a redeploy, which defeats the purpose of alias-based promotion. It also abandons the alias indirection that lets promotions happen without touching code.
- C. Copying prompt text into a separate file bypasses the Prompt Registry entirely, losing version history, commit messages, and Unity Catalog governance. It also does not integrate with the alias-based loading mechanism the application already uses.
- D. Storing the prompt string directly in endpoint environment variables bypasses the registry's versioning and audit trail, and would still require reconfiguring and restarting the endpoint. This discards the governance benefits the registry is designed to provide.
3.Testing agent components, from unit to canary
Testing an agent works best in layers, with each layer covering a narrower part of the system than the end-to-end flow. Unit tests check business logic in isolation, for example with pytest. databricks bundle validate checks configuration and catches mismatched resource references, invalid permissions and syntax errors before deploy. A local run catches problems before deployment: start the agent server locally, send it sample requests, and confirm that MLflow traces appear. Trace autologging makes each step the agent takes visible. Clear tool and parameter descriptions help the agent call its tools correctly.
Load testing isolates one component as well. Databricks recommends a ramp-to-saturation test against a build of the agent with a mock LLM, which separates the throughput of the Apps infrastructure from model latency.
The last layer runs after deployment. databricks bundle run returns once the agent has been told to start, but the agent process can still crash while booting. Add a smoke-test step after the health check that posts a canary request to /invocations.
- name: Smoke test invocations
env:
APP_NAME: my-agent
run: |
APP_URL=$(databricks apps get "$APP_NAME" --output json | jq -r '.url')
TOKEN=$(databricks auth token | jq -r '.access_token')
STATUS=$(curl -sS -o /tmp/canary.json -w "%{http_code}" \
-X POST "$APP_URL/invocations" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"input": [{"role": "user", "content": "ping"}], "stream": false}')
if [ "$STATUS" != "200" ]; then
echo "Smoke test failed with status $STATUS:" >&2
cat /tmp/canary.json >&2
exit 1
fi
echo "Smoke test passed."Checkpoint 4 of 4· Check yourself
You want to know the maximum QPS the Databricks Apps hosting layer can sustain for your agent, without model latency distorting the result. Which test fits?
Replacing the LLM with a mock removes model latency, so the load test measures only the throughput of the Apps infrastructure.
“Run a ramp-to-saturation load test against a mock-LLM build of your agent to isolate Databricks Apps infrastructure throughput from model latency.”Source: docs.databricks.com
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.If databricks bundle run succeeds, the agent is up and serving requests.Why is that wrong?
bundle run returns once the agent has been signalled to start, and the agent can still fail while booting. A canary request to /invocations confirms that it is healthy.
2.A personal access token stored as a CI secret is fine for the smoke-test request.Why is that wrong?
Databricks Apps accept only OAuth tokens for invocation. Get a workspace OAuth token from databricks auth token.
3.Every Delta Sync index picks up source-table changes automatically.Why is that wrong?
Only CONTINUOUS mode processes new data as it arrives. A TRIGGERED index refreshes once per update and needs an explicit sync.
Practise it for real
Run one deploy cycle for an agent app by hand and refresh its TRIGGERED Delta Sync index, using the same commands your CI workflow will run.
1.Run databricks bundle validate in the agent template directory.
Why: Catches mismatched resource references, invalid permissions and syntax errors before anything is deployed.
You should see: Validation passes with no configuration errors.
2.Run databricks bundle deploy --target prod.
Why: Pushes the bundle's code and configuration to the target workspace.
You should see: The deploy succeeds. If it fails with a Terraform type error, commit a numeric experiment_id in databricks.yml.
3.Run databricks bundle run <bundle-key> --target prod.
Why: Starts the app. It returns before the agent has finished booting.
You should see: The command returns once the start has been signalled.
4.Run databricks vector-search-indexes sync-index <your-delta-sync-index>.
Why: A TRIGGERED index only reflects source-table changes after a sync.
You should see: Synchronization is triggered for the Delta Sync index.
5.Run the smoke-test commands from deploy.yml against your app's /invocations URL with a token from databricks auth token.
Why: Confirms that the agent actually serves a request after boot.
You should see: The request returns HTTP 200 and the script prints "Smoke test passed."
Stuck? Get a nudge
If the curl call is rejected, check that you are using the OAuth token from databricks auth token and not a personal access token.
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.https://docs.databricks.com/aws/en/dev-tools/ci-cdOfficial docs
“Declarative Automation Bundles are the recommended approach to CI/CD on Databricks.”
↩︎ The agent's CI/CD pipeline with Declarative Automation Bundles - 2.
“Maintain separate workspaces for development, staging, and production.”
↩︎ The agent's CI/CD pipeline with Declarative Automation Bundles“Implement unit tests for business logic using libraries, such as pytest for Python and ScalaTest for Scala.”
↩︎ Testing agent components, from unit to canary - 3.
“every merge to your main branch deploys and restarts your agent on Databricks Apps.”
↩︎ The agent's CI/CD pipeline with Declarative Automation Bundles“If experiment_id is empty, databricks bundle deploy fails with a Terraform type error”
↩︎ The agent's CI/CD pipeline with Declarative Automation Bundles“databricks bundle run returns as soon as the runner signals the agent to start, but the agent process may still fail during boot.”
↩︎ Exam trap 1“Databricks Apps only accept OAuth tokens for invocation.”
↩︎ Exam trap 2 - 4.
“The primary key column and embedding source column or embedding vector column are always synced.”
↩︎ Updating the Vector Search index in the pipeline“CONTINUOUS: The pipeline processes new data as it arrives in the source table to keep the vector index fresh.”
↩︎ Exam trap 3“TRIGGERED: The system stops processing after successfully refreshing the source table in the pipeline once”
↩︎ Prediction - 5.https://docs.databricks.com/aws/en/dev-tools/cli/reference/vector-search-indexes-commandsOfficial docs
“Name of the vector index where data is to be upserted. Must be a Direct Vector Access Index.”
↩︎ Updating the Vector Search index in the pipeline“Name of the vector index to synchronize. Must be a Delta Sync Index.”
↩︎ Checkpoint - 6.
“Use local development to catch issues before you deploy.”
↩︎ Testing agent components, from unit to canary“Clear tool and parameter descriptions ensure your agent understands your tools and uses them appropriately.”
↩︎ Testing agent components, from unit to canary“Run databricks bundle validate before you deploy to catch YAML configuration issues early.”
↩︎ Checkpoint
Also cited
“Run a ramp-to-saturation load test against a mock-LLM build of your agent to isolate Databricks Apps infrastructure throughput from model latency.”
↩︎ Checkpoint