CertSafari
    Databricks Certified Machine Learning Associate· Lessons

    Domain 4 · Lesson 44/48

    Deploy a custom model to a Model Serving endpoint

    Deploy a custom model to a model endpoint

    22 min read
    2% of exam
    6 sources
    Published 22 Oct 2026
    Docs as of 17 Oct 2026

    What you will be able to do

    • Package custom preprocessing and prediction logic as an mlflow.pyfunc.PythonModel that Model Serving can host
    • Validate a logged model's dependencies and input with mlflow.models.predict before deployment
    • Create a serving endpoint for a Unity Catalog model version through the Serving UI, the REST API or the MLflow Deployments SDK
    • Choose a workload type, workload size, provisioned concurrency and scale-to-zero setting for a given traffic pattern
    • Roll a live endpoint to a new model version, or split traffic with a challenger, without downtime

    Key concept

    Served entity — A served entity is one Unity Catalog model version in an endpoint's configuration, together with the compute it runs on: workload type, workload size or concurrency range, and the scale-to-zero setting. Creating, resizing, updating and traffic-splitting an endpoint all come down to editing its list of served entities.

    1.What a Model Serving endpoint gives a custom model

    A Model Serving endpoint turns a registered model into an API that other systems call over HTTP. Databricks uses the term custom model for anything you train or write yourself, as opposed to foundation models and external models. That covers a scikit-learn classifier, an XGBoost booster, a Hugging Face transformer, or plain Python with no ML library at all. All of these deploy the same way.

    Every custom model takes the same path to an endpoint:

    1. Log it in MLflow format, using a built-in flavor such as mlflow.sklearn or a custom pyfunc wrapper.
    2. Register it, preferably in Unity Catalog.
    3. Create a serving endpoint whose configuration points at one registered version.

    The endpoint is a managed REST service that autoscales on serverless compute. The calling application sends a request and waits for the prediction in the response. Use this pattern when each event needs its own answer right away, such as a payment service that must score each transaction before approving it.

    Batch and streaming scoring don't meet that requirement, even with the same registered model. A nightly job that loads the model with mlflow.pyfunc.load_model and scores a DataFrame produces results hours after each transaction. A pipeline flow writes its scores to a table, so the service that is waiting never gets a response. A notebook rerun by hand is not a service at all. Batch, real-time and streaming serving trade-offs compares the three patterns. For a per-request latency budget measured in milliseconds, the answer is an endpoint called synchronously.

    Checkpoint 1 of 9· Put it in order

    A team has trained a scikit-learn fraud classifier and needs a payment service to score each transaction over HTTP. Put the steps for getting the model there in order.

    1. 1.Create a Model Serving endpoint whose configuration points at one registered model version
    2. 2.Log the model in MLflow format with a built-in flavor such as mlflow.sklearn, or with a custom pyfunc wrapper
    3. 3.Have the payment service send each transaction to the endpoint's REST API and wait for the prediction
    4. 4.Register the logged model, preferably in Unity Catalog

    Checkpoint 2 of 9· Exam question

    A team has trained a custom scikit-learn fraud-detection model and registered it in Unity Catalog. The application must return a prediction within 50 milliseconds of each transaction as it happens, and transactions arrive continuously throughout the day. Which approach should the team use to serve predictions?

    Sources12

    2.Packaging custom logic as a pyfunc model

    Use mlflow.pyfunc.PythonModel when a built-in flavor can't express what the endpoint has to do. Typical cases are:

    • inputs that need preprocessing before they reach the model
    • a framework MLflow doesn't support natively
    • raw outputs that need post-processing
    • branching logic that depends on the request
    • code that isn't a trained model at all

    A PythonModel splits its work between two methods, and where you put each piece of work affects latency. load_context runs once when the model loads, so weights, tokenizers and lookup tables belong there. predict runs on every request and should contain only per-request work. If you load a large file inside predict, every request pays that cost again. Shared helper modules ship with the model through code_path and can be imported inside load_context.

    A custom pyfunc: heavy objects load once in load_context; predict formats input, scores and post-processes on every requestpython
    class CustomModel(mlflow.pyfunc.PythonModel):
        def load_context(self, context):
            self.model = torch.load(context.artifacts["model-weights"])
            from preprocessing_utils.my_custom_tokenizer import CustomTokenizer
            self.tokenizer = CustomTokenizer(context.artifacts["tokenizer_cache"])
    
        def format_inputs(self, model_input):
            # insert some code that formats your inputs
            pass
    
        def format_outputs(self, outputs):
            predictions = (torch.sigmoid(outputs)).data.numpy()
            return predictions
    
        def predict(self, context, model_input):
            model_input = self.format_inputs(model_input)
            outputs = self.model.predict(model_input)
            return self.format_outputs(outputs)

    During deployment, Databricks builds a container from the logged model. Anything the model imports must be listed in the model, because the base image covers only some system-level packages. A signature, input example and pip_requirements are attached when you log the model, as covered in Model signature, input example and pip requirements. For serving, two of these matter most. Unity Catalog won't register a model without a signature, and the listed packages must install cleanly in the container. Built-in flavors capture their dependencies automatically. For a custom pyfunc you add them explicitly, with pip_requirements, extra_pip_requirements or conda_env.

    Databricks Runtime ML has a specific trap here. It ships mlflow-skinny instead of the full mlflow package. If you log a pyfunc there without pip_requirements, mlflow-skinny is written to the model's conda.yaml, and Model Serving can't build the image from it. The fix is to pin mlflow==<version> yourself.

    Pinning full mlflow when logging a pyfunc on Databricks Runtime MLpython
    mlflow.pyfunc.log_model(
        name="model",
        python_model=your_model,
        pip_requirements=["mlflow==3.8.1"],  # use mlflow, not mlflow-skinny
        registered_model_name="catalog.schema.model_name",
    )

    Endpoint deployment is a slow way to find a missing package. Run mlflow.models.predict against the logged model instead. With the default env_manager="virtualenv", it rebuilds the model's dependencies in an isolated environment that simulates serving. A missing or conflicting package then fails in your notebook within minutes, rather than after a deployment that can take ten minutes or more. pip_requirements_override lets you try other package versions without logging the model again. Once the right versions are known, update_model_requirements writes them into the logged model in place.

    Testing a prediction in a virtualenv that simulates the serving environmentpython
    mlflow.models.predict(
      model_uri=model_uri,
      input_data={"col1": 34.2, "col2": 11.2, "col3": "green"},
      content_type="json",
      env_manager="virtualenv",
      install_mlflow=False,
      pip_requirements_override=["pillow==10.3.0", "scipy==1.13.0"],
    )

    Checkpoint 3 of 9· Check yourself

    A data scientist on Databricks Runtime ML logs a custom pyfunc model without pip_requirements and registers it in Unity Catalog. Deploying it to a serving endpoint fails because the container image can't be built. What is the most likely cause, and what fixes it?

    Checkpoint 4 of 9· Check yourself

    Every request to a custom pyfunc endpoint is slow. Its predict method calls torch.load on a large weights file before scoring. Which change addresses the cause?

    Sources314

    3.Creating the endpoint and its recorded creator

    There are three ways to create an endpoint, and they accept the same configuration:

    • the Serving UI (Serving > Create serving endpoint)
    • the REST API (POST /api/2.0/serving-endpoints)
    • the MLflow Deployments SDK (get_deploy_client("databricks").create_endpoint(...))

    The configuration is a served_entities list. Each entry names a model with its full three-level Unity Catalog name in entity_name and gives the version in entity_version. Endpoint names may not start with databricks-, because that prefix is reserved for Databricks' own endpoints.

    REST body for POST /api/2.0/serving-endpoints: version 3 of a Unity Catalog model, with a concurrency range and scale to zero offjson
    {
      "name": "uc-model-endpoint",
      "config":
      {
        "served_entities": [
          {
            "name": "ads-entity",
            "entity_name": "catalog.schema.my-ads-model",
            "entity_version": "3",
            "min_provisioned_concurrency": 4,
            "max_provisioned_concurrency": 12,
            "scale_to_zero_enabled": false
          }
        ]
      }
    }

    A new endpoint starts as Not Ready. Databricks first packages the model and its environment and then provisions compute, which takes about ten minutes and longer for large or dependency-heavy models. When it finishes, the response shows state.ready as READY and the served entity's deployment as DEPLOYMENT_READY. The response also contains a creator field, which has consequences for the rest of the endpoint's life.

    Whoever calls create becomes the endpoint's recorded creator, permanently. The endpoint uses that identity to read Unity Catalog resources. Model grants work as described in Unity Catalog privileges on registered models. For serving, the recorded creator needs USE CATALOG on the catalog, USE SCHEMA on the schema and EXECUTE on the model. If the model declares function dependencies, it also needs EXECUTE on those functions. These grants are checked when the endpoint is created and again on every update. If one is missing, the request fails with PERMISSION_DENIED, even when the person submitting the update has every permission.

    The creator can't be changed, so a revoked grant or a deactivated account leaves the endpoint unable to accept updates. The fix is to delete the endpoint and recreate it under a service principal that holds the grants and is a current workspace member. To avoid this, create production endpoints as a long-lived service principal owned by the team, not as a personal user.

    Checkpoint 5 of 9· Fill the gap

    This REST body creates an endpoint that serves version 3 of a model registered in Unity Catalog. What value goes in the blank?

    {
      "name": "uc-model-endpoint",
      "config":
      {
        "served_entities": [
          {
            "name": "ads-entity",
            "entity_name": " ? ",
            "entity_version": "3",
            "min_provisioned_concurrency": 4,
            "max_provisioned_concurrency": 12,
            "scale_to_zero_enabled": false
          }
        ]
      }
    }

    Sources51

    4.Workload type, concurrency and scale to zero

    Each served entity gets compute along two settings. The workload type sets the hardware and memory per unit of concurrency. Workload size or a provisioned-concurrency range sets how many requests the entity can process at the same time. Choose the hardware first.

    Workload types for custom model serving and what each provides
    workload_typeHardwareMemoryChoose it when
    CPUStandard CPU4GB per concurrencyTypical scikit-learn or XGBoost models
    CPU_MEDIUMSame CPU hardware8GB per concurrencyThe model needs more memory than CPU gives; you accept less concurrency per worker
    CPU_LARGESame CPU hardware16GB per concurrencyThe model needs even more memory per worker
    GPU_SMALL1xT416GB per concurrencyThe code actually runs inference on the GPU
    GPU_MEDIUM1xA10G24GB per concurrencyA larger GPU model

    A GPU workload costs something in three places, and the model has to be able to use the GPU for that cost to pay off. Your code must place inference on the GPU; MLflow does this automatically only for the PyTorch and Transformers flavors. GPU container builds take longer than CPU builds, and GPU endpoints autoscale more slowly than CPU endpoints. A plain regression model that never puts work on a GPU gets the slower builds and autoscaling with nothing in return.

    Size concurrency from traffic, not from a guess. Estimate it as queries per second × model execution time in seconds. At 20 QPS and 0.3 s per prediction, that is 6 requests in flight. You can set the size in two ways. workload_size picks a preset range: Small handles 0-4 concurrent requests, Medium 8-16 and Large 16-64. Alternatively, set min_provisioned_concurrency and max_provisioned_concurrency, which must both be multiples of 4. The endpoint autoscales within that range. It scales up almost immediately when traffic rises and scales down every five minutes. On GPU workloads, concurrency also sets the number of replicas, which is concurrency divided by 4, so a minimum of 12 provisions 3 replicas.

    Scale to zero (scale_to_zero_enabled) releases all compute after 30 minutes of inactivity. While no traffic arrives, the endpoint costs nothing. The first request afterwards hits a cold start, usually 10-20 seconds and sometimes minutes, and there is no SLA on how long it takes. GPU capacity in particular isn't guaranteed to come back promptly. Use scale to zero for development endpoints and for models queried only occasionally, where a slow first request is acceptable. Leave it off for any production endpoint with steady traffic or a latency target. For steady moderate traffic, size the entity for its actual concurrency, for example Medium, and keep it running.

    Checkpoint 6 of 9· Match them up

    Match each workload_type to the served model it fits best.

    Tap a term, then the definition that fits it.

    Checkpoint 7 of 9· Exam question

    A served entity hosts a custom regression model behind a production endpoint that receives a steady, moderate volume of prediction requests around the clock, with no long idle periods. Which configuration choice best fits this workload?

    Sources15

    5.Updating a live endpoint to a new model version

    To move an endpoint to a new model version, you change its served entities. You never delete and recreate it. In the UI this is Edit endpoint. Through the API it is PUT /api/2.0/serving-endpoints/{name}/config, and in the MLflow Deployments SDK it is client.update_endpoint_config(...), which takes the same parameters as the REST API. You can change most of the configuration this way, including entity versions, compute and traffic, but not the endpoint name.

    Part of a PUT .../config body that rolls the endpoint to version 5 and routes all traffic to itjson
        "served_entities": [
          {
            "entity_name": "catalog.schema.my-ads-model",
            "entity_version": "5",
            "workload_size": "Small",
            "scale_to_zero_enabled": true
          }
        ],
        "traffic_config":
        {
          "routes": [
            {
              "served_model_name": "my-ads-model-5",
              "traffic_percentage": 100
            }
          ]
        }

    Updates don't cause downtime. Version 4 keeps serving every request until version 5 is ready, and only then does traffic move to the new version. You pay for that safety: until the switch completes, both configurations are billed. If the update fails, for example because a package won't install or the recorded creator has lost EXECUTE, the old configuration stays active as if nothing had happened. A failed update can therefore look like a successful one. Check the endpoint's state after every update. Only one update can run at a time, and you can cancel an update in progress from the endpoint's page in the Serving UI.

    The same update mechanism supports a challenger. List both versions as served entities on one endpoint, and add a traffic_config whose routes give each entity a traffic_percentage, for example 90 for the current version and 10 for the challenger. Live traffic is then split so the two can be compared on the same requests, while production continues to run on the incumbent. Two separate endpoints with the client always calling one of them never tests the challenger on live traffic. Replacing the incumbent outright takes away the safe default. Routing options and limits are covered in splitting traffic between served entities. After deployment, the application sends requests as described in deploying and querying a model for real-time inference.

    Checkpoint 8 of 9· Fill the gap

    An endpoint is already live. This MLflow Deployments SDK code changes its served entity and traffic routes in place, without deleting or recreating the endpoint. Which method goes in the blank?

    endpoint = client. ? (
      endpoint=f"{endpointname}",
      config={
        "served_entities": [
            {
                "entity_name": f"{catalog}.{schema}.{model_name}",
                "entity_version": "1",
                "workload_size": "Small",
                "scale_to_zero_enabled": True
            }
        ],
        "traffic_config": {
            "routes": [
                {
                    "served_model_name": f"{model_name}-1",
                    "traffic_percentage": 100
                }
            ]
        }
      }

    Checkpoint 9 of 9· Exam question

    An ML engineer wants to compare a new challenger model version against the current production model on the same endpoint, sending most live traffic to the current version while a small share reaches the challenger for evaluation. Which endpoint configuration achieves this?

    Sources156

    Practise it for real

    Take a custom pyfunc model from logging to a live endpoint, then roll the endpoint to a second version without downtime.

    1. 1.Log a PythonModel with mlflow.pyfunc.log_model, passing pip_requirements that pin the full mlflow package and registered_model_name set to a catalog.schema.model_name in Unity Catalog.

      Why: Databricks Runtime ML would otherwise record mlflow-skinny, and the serving container would fail to build.

      You should see: Version 1 of the model appears in Unity Catalog with a signature.

    2. 2.Run mlflow.models.predict on the model URI with an input example and env_manager="virtualenv".

      Why: It rebuilds the logged dependencies in an isolated environment that simulates serving, so dependency errors show up before you deploy.

      You should see: Predictions are printed to stdout with no install or import errors.

    3. 3.Call get_deploy_client("databricks").create_endpoint with one served entity for version 1, workload_size Small, and scale_to_zero_enabled True for this test endpoint.

      Why: This creates the endpoint and records you, or the service principal you run as, as its permanent creator.

      You should see: The endpoint is listed as Not Ready, then READY after about ten minutes.

    4. 4.Log and register version 2, then call update_endpoint_config with a served entity for version 2 and a traffic_config route sending 100 percent to it.

      Why: Updating the served entities rolls the endpoint to the new version while version 1 keeps serving.

      You should see: The endpoint's configuration update state stays in progress until version 2 is ready, then the endpoint returns to NOT_UPDATING with version 2 serving.

    Stuck? Get a nudge

    If the update fails with PERMISSION_DENIED, check the endpoint's recorded creator, not your own grants.

    Sources

    (6)

    Every claim above is drawn from one of these pages, quoted as it was written on the date shown.

    1. 1.
      “Model Serving can deploy any Python model or custom code as a production-grade API using CPU or GPU compute resources.”
      ↩︎ What a Model Serving endpoint gives a custom model
      “application-level dependencies must be explicitly specified in your MLflow model.”
      ↩︎ Packaging custom logic as a pyfunc model
      “Signatures are necessary for logging models to the Unity Catalog.”
      ↩︎ Packaging custom logic as a pyfunc model
      “This process can take approximately 10 minutes, but may take longer depending on model complexity, size, and dependencies.”
      ↩︎ Creating the endpoint and its recorded creator
      “The CPU_MEDIUM and CPU_LARGE workload types let you trade concurrency for more memory per worker on the same CPU hardware.”
      ↩︎ Workload type, concurrency and scale to zero
      “you must make sure that your code is set up so that predictions are run on the GPU”
      ↩︎ Workload type, concurrency and scale to zero
      “Autoscaling for GPU serving takes longer than for CPU serving.”
      ↩︎ Workload type, concurrency and scale to zero
      “provisioned concurrency = queries per second (QPS) * model execution time (s)”
      ↩︎ Workload type, concurrency and scale to zero
      “Endpoints scale up almost immediately with increased traffic and scale down every five minutes to match reduced traffic.”
      ↩︎ Workload type, concurrency and scale to zero
      “allows them to scale down to zero after 30 minutes of inactivity.”
      ↩︎ Workload type, concurrency and scale to zero
      “If model computation takes longer than 597 seconds, requests will time out.”
      ↩︎ Workload type, concurrency and scale to zero
      “Databricks performs a zero-downtime update of endpoints by keeping the existing endpoint configuration up until the new one becomes ready.”
      ↩︎ Updating a live endpoint to a new model version
      “you are billed for both the old and new endpoint configurations until the transition is complete.”
      ↩︎ Updating a live endpoint to a new model version
      “Scaling up from zero usually takes 10-20 seconds, but can sometimes take minutes. There is no SLA on scale from zero latency.”
      ↩︎ Exam trap 1
      “After the model is logged, register it in the Unity Catalog (recommended) or the workspace registry.”
      ↩︎ Checkpoint
    2. 2.
      “Model Serving provides low-latency, high-uptime managed REST endpoints with serverless autoscaling.”
      ↩︎ What a Model Serving endpoint gives a custom model
    3. 3.
      “load_context - anything that needs to be loaded just one time for the model to operate should be defined in this function.”
      ↩︎ Packaging custom logic as a pyfunc model
      “predict - this function houses all the logic that is run every time an input request is made.”
      ↩︎ Packaging custom logic as a pyfunc model
      “Model Serving requires mlflow (not mlflow-skinny) in conda.yaml and cannot build the container image otherwise.”
      ↩︎ Packaging custom logic as a pyfunc model
      “Model Serving requires mlflow (not mlflow-skinny) in conda.yaml and cannot build the container image otherwise.”
      ↩︎ Exam trap 3
      “anything that needs to be loaded just one time for the model to operate should be defined in this function”
      ↩︎ Checkpoint
    4. 4.
      “MLflow rebuilds your model's logged dependencies in an isolated environment that simulates the serving runtime”
      ↩︎ Packaging custom logic as a pyfunc model
    5. 5.
      “Endpoint names cannot use the databricks- prefix.”
      ↩︎ Creating the endpoint and its recorded creator
      “USE CATALOG on the catalog, USE SCHEMA on the schema, EXECUTE on the model”
      ↩︎ Creating the endpoint and its recorded creator
      “you must delete the endpoint and recreate it under a service principal that has the required permissions and is a current workspace member.”
      ↩︎ Creating the endpoint and its recorded creator
      “Use a long-lived service principal owned by your team as the endpoint creator.”
      ↩︎ Creating the endpoint and its recorded creator
      “Available sizes are Small for 0-4 requests, Medium 8-16 requests, and Large for 16-64 requests.”
      ↩︎ Workload type, concurrency and scale to zero
      “Concurrency values must be multiples of 4.”
      ↩︎ Workload type, concurrency and scale to zero
      “The number of replicas equals the concurrency value divided by 4.”
      ↩︎ Workload type, concurrency and scale to zero
      “Scale to zero is not recommended for production endpoints, as capacity is not guaranteed when scaled to zero.”
      ↩︎ Workload type, concurrency and scale to zero
      “While there is an update in progress, another update cannot be made.”
      ↩︎ Updating a live endpoint to a new model version
      “Updates to the endpoint configuration can fail.”
      ↩︎ Updating a live endpoint to a new model version
      “You can serve multiple models or model versions from a single endpoint and control the traffic split between them.”
      ↩︎ Key concept
      “Databricks records the calling identity as the endpoint's creator.”
      ↩︎ Exam trap 2
      “Until the new configuration is ready, the old configuration keeps serving prediction traffic.”
      ↩︎ Exam trap 4
      “Updates fail with PERMISSION_DENIED if the recorded creator is no longer a workspace member, even when the caller has valid permissions.”
      ↩︎ Prediction
    6. 6.
      “split traffic between different models to compare their performance and facilitate A/B testing”
      ↩︎ Updating a live endpoint to a new model version

    Ready to test yourself?

    Practise the 7 questions on this subdomain.

    Spotted a mistake, or was something unclear? Tell us.