Defining ML Assets in Databricks Asset Bundles (DABs)

1.The anatomy of a bundle

Declarative Automation Bundles (formerly called Databricks Asset Bundles, or DABs) manage Databricks projects as infrastructure-as-code. A bundle is the complete definition of a project. It holds the source files with your business logic, plus the definitions and settings for the Databricks resources that run them. For ML projects, those resources include Model Serving endpoints, MLflow experiments and MLflow registered models, alongside jobs and pipelines. Everything is written in YAML and kept next to the code, so it gets the same source control, code review and CI/CD.

Every bundle has exactly one main configuration file, databricks.yml, at the root of the project. It can pull in other configuration files, such as per-resource YAML files, through the include mapping. The only required top-level mapping is bundle with a name. In practice you also add targets, the environments you deploy to. Other top-level mappings in the specification include run_as, include, scripts, sync, artifacts, variables, workspace, permissions and resources.

bundle:
  name: my_bundle

targets:
  dev:
    default: true

Sources12

2.Declaring experiments, registered models and serving endpoints

Each resource in the resources mapping maps to a Databricks REST API object, and the keys it accepts are that API's create-request fields written as YAML. In an experiment declaration, the keys follow the POST /api/2.0/mlflow/experiments/create payload. In a model declaration, they follow the Unity Catalog POST /api/2.1/unity-catalog/models payload. If you already have an asset in the workspace, databricks bundle generate writes the YAML for it.

resources:
  experiments:
    experiment:
      name: /Workspace/Users/someone@example.com/my_experiment
      permissions:
        - level: CAN_READ
          group_name: users
      tags:
        - key: mlflow.note.content
          value: MLflow experiment used to track runs

resources:
  registered_models:
    model:
      name: my_model
      catalog_name: ${bundle.target}
      schema_name: mlops_schema
      comment: Registered model in Unity Catalog for ${bundle.target} deployment target
      grants:
        - privileges:
            - EXECUTE
          principal: account users

A serving endpoint's config lists the served_entities it serves. Each entry names a Unity Catalog model by its full catalog.schema.model name and can set entity_name, entity_version, workload_size, scale_to_zero_enabled, workload_type and environment_vars. traffic_config.routes sets what percentage of requests each served model receives. The older served_models field is deprecated.

resources:
  model_serving_endpoints:
    uc_model_serving_endpoint:
      name: 'uc-model-endpoint'
      config:
         ? :
          - entity_name: 'myCatalog.mySchema.my-ads-model'
            entity_version: '10'
            workload_size: 'Small'
            scale_to_zero_enabled: 'true'
        traffic_config:
          routes:
            - served_model_name: 'my-ads-model-10'
              traffic_percentage: '100'

Sources34

3.Targets: one definition, many environments

The targets mapping is where a bundle meets the dev/staging/prod design. Each target can set its own workspace (for example a different host), resources, variables, permissions, run_as and mode. Exactly one target can be marked default: true. Databricks recommends setting host instead of relying on default wherever possible, because it makes the configuration more portable.

You never have to copy a whole resource into each target. A target lists only the fields it changes, and every other field comes from the top-level resources mapping. In the example below, prod points at a different workspace and swaps the cluster ID, while notebook_task still comes from the top-level definition. The same pattern would let a serving endpoint run with a different workload size in prod.

targets:
  dev:
    default: true
  prod:
    workspace:
      host: https://<production-workspace-url>
    resources:
      jobs:
        hello-job:
          name: hello-job
          tasks:
            - task_key: hello-task
              existing_cluster_id: 2345-678901-fabcd456

Variables are the second way to vary settings. You declare them in the top-level variables mapping with a description and a default, refer to them inside resources, and give a different value in each target's variables block.

Deployment modes are optional presets on a target. mode: development prefixes every resource that is not deployed as a file or notebook with [dev ${workspace.current_user.short_name}] and adds a dev tag to deployed jobs and pipelines. That way several developers can deploy the same bundle into one dev workspace without clashing.

targets:
  dev:
    mode: development

Sources25

4.Validate, deploy, run, destroy

The Databricks CLI runs a bundle through the same four commands in every environment. databricks bundle validate checks the configuration and warns about resource properties it does not recognise. databricks bundle deploy -t <target> sends the resources and artifacts to that target's workspace and leaves other targets alone. After deployment, jobs run on their own schedules. databricks bundle run -t <target> <job-name> starts a job immediately, for example an MLOps Stacks model_training_job. databricks bundle destroy -t <target> removes what was deployed, after asking you to confirm. If you leave out -t, the CLI uses the default target, so deploying to any other target requires the flag.

# You must specify "-t prod", because the "dev" target
# is already set to "default: true":
databricks bundle validate
databricks bundle deploy -t prod
databricks bundle run -t prod hello_job

Sources43

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.A bundle's model serving endpoint lists its models under served_models.Why is that wrong?

   served_entities is the current field. served_models is deprecated.

   Covered in Declaring experiments, registered models and serving endpoints

2. 2.To change one setting in prod, you must repeat the whole resource definition inside the prod target.Why is that wrong?

   A target only needs the fields it changes. Everything else comes from the top-level resources mapping.

   Covered in Targets: one definition, many environments

Practise it for real

Create an MLOps Stacks bundle, deploy it to a dev target, run its training job, and clean up

1. 1.Run databricks bundle init mlops-stacks and choose CICD_and_Project at the first prompt

   Why: This creates both the ML code and the CI/CD components, so you can see the full project structure

   You should see: Starter files for the project, including databricks.yml, appear in your working directory

2. 2.From the project root, run databricks bundle validate

   Why: This catches configuration errors and unknown resource properties before anything is deployed

   You should see: A summary of the bundle configuration. If errors appear, fix them and validate again

3. 3.Run databricks bundle deploy -t dev

   Why: This deploys the resources and artifacts to the dev target only, leaving staging and prod untouched

   You should see: The project's jobs and other resources are created in the dev workspace and will run on their schedules

4. 4.Run databricks bundle run -t dev model_training_job

   Why: This starts training now instead of waiting for the next scheduled run

   You should see: The CLI prints a link to the job run in the Databricks UI

5. 5.Run databricks bundle destroy -t dev and confirm when prompted

   Why: This removes the resources and artifacts the dev deployment created

   You should see: After you confirm, the deployed resources are deleted from the dev workspace