CertSafari
    Snowflake SnowPro Advanced: MLOps Engineer (MLA-B01)· Lessons

    Domain 3 · Lesson 10/17

    Real-time model inference on SPCS: deploy, call, scale and monitor

    Implement inference deployment patterns.

    17 min read
    6% of exam
    6 sources
    Published 20 Oct 2026
    Docs as of 18 Oct 2026

    What you will be able to do

    • Choose between warehouse batch, real-time SPCS service and SPCS job inference for a workload, and explain how target_platforms limits where a model can run
    • Run registry-backed batch inference on a dedicated warehouse from Python and SQL, and size the warehouse for the model
    • Deploy a registered model version as an HTTPS inference service with create_service and pick its compute pool, GPU and ingress settings
    • Find a service's public and internal endpoints, build the URL for a model method and call it with a PAT
    • Set min_instances, max_instances and num_workers so a service scales without cold starts or out-of-memory failures
    • Read latency, request-rate and resource metrics for a real-time inference service

    Key concept

    Two engines, three inference patterns — Every registered model runs on one of two engines: a virtual warehouse (the SQL engine) or Snowpark Container Services. On top of them sit three patterns: SQL batch inference on a warehouse, a long-running SPCS service with an HTTPS endpoint, and a one-off SPCS batch job. Which one you pick depends on latency, data type and scale.

    1.Three inference patterns across two compute engines

    Inference in Snowflake runs on one of two compute engines: the virtual warehouse, which is the SQL engine, or Snowpark Container Services (SPCS). The Model Registry gives you one interface to both, so the same logged model version can be scored from SQL, from Python or over HTTP. The real decision is which of three patterns fits the workload.

    The three inference patterns and the workload each one serves
    AspectReal-time inference (SPCS)Native batch inference (SQL)Job-based batch (SPCS)
    Primary goalLow-latency, sub-second responses for live usersEmbedding models in SQL data pipelinesLarge-scale, decoupled compute for unstructured data
    Best forWeb and mobile app backends; high-concurrency request spikesDynamic Tables, Snowpark, dbt; SQL-first analysts and data engineersImages, video and audio files; large historical backfills; multimodal data
    Data sourceSmall inputs passed in the HTTP payloadData in Snowflake tablesData in Snowflake stages (files)
    ScalingHorizontal autoscaling to match request volumeServerless scaling through virtual warehousesHigh-throughput distributed processing of bulk data

    The latency requirement usually settles the choice. A mobile app that needs a fraud score in well under a second, on traffic that rises and falls during the day, needs a real-time service: a model hosted on an SPCS compute pool behind an HTTPS endpoint that autoscales horizontally. Running MODEL(...)!predict in a SQL statement for each request puts a warehouse query on every call. Starting a run_batch job per request means building an image, provisioning a job and writing Parquet to a stage before the caller gets anything back. Both are batch patterns, and neither fits an interactive call path.

    Where a model is allowed to run is fixed when you log it. The target_platforms argument of log_model lists the engines the model is prepared for, and you must include WAREHOUSE there before the model can run inside a warehouse. For a model that is already logged, run SHOW VERSIONS IN MODEL and look at the runnable_in column. A warehouse can't host a model that needs a GPU, and it struggles with large models. Those models go to SPCS, where the same version can serve as an online service or run as a batch job.

    Registry-backed batch execution on a warehouse runs a model version fetched from the Model Registry, so the model never leaves Snowflake and the data never leaves the warehouse. From Python, mv.run(input_features, function_name="predict") on a ModelVersion scores a Snowpark or pandas DataFrame and returns the same DataFrame type it was given. A Snowpark result is lazy: nothing executes until collect, show or to_pandas is called. In SQL, SELECT MODEL(my_model,LAST)!predict(...) FROM my_table does the same over a table, and the alias or version in the call decides which registered version scores the rows.

    The warehouse that runs the call is the capacity you are spending, so a large scoring pass belongs on a warehouse of its own, sized for the model. Model size is the limit to watch: a model above 15GB can't run on a warehouse, and the ceiling is lower on smaller sizes. A scikit-learn or XGBoost model scoring millions of table rows fits comfortably. A model that needs a GPU, or a heavy deep-learning model, has to go to SPCS instead.

    Checkpoint 1 of 8· Match them up

    Match each inference pattern to the workload it fits best

    Tap a term, then the definition that fits it.

    Sources12

    2.Deploying a model version as an HTTPS service

    create_service on a ModelVersion object turns a registered model into a managed HTTP server on SPCS. Snowflake builds the container image and runs it on the compute pool you name, so you don't manage Docker images or Kubernetes. To deploy, you need USAGE or OWNERSHIP on the compute pool, OWNER or READ on the model, and the account-level BIND SERVICE ENDPOINT privilege if the service will have a public endpoint.

    Getting a model version and deploying it as a service with a public endpointpython
    # reg is a snowflake.ml.registry.Registry object
    example_mv_object = reg.get_model("mymodel_name").version("version_name") # a snowflake.ml.model.ModelVersion object
    
    example_mv_object.create_service(service_name="myservice",
                      service_compute_pool="my_compute_pool",
                      ingress_enabled=True,
                      gpu_requests=None)
    What each create_service argument controls
    ArgumentEffect
    service_nameName of the service; must be unique in the account
    service_compute_poolExisting pool that runs the model; SYSTEM_COMPUTE_POOL_CPU or SYSTEM_COMPUTE_POOL_GPU can be used if the model fits
    ingress_enabledMust be True to call the service from outside Snowflake
    gpu_requestsNumber of GPUs; decides CPU or GPU execution for models that support both
    num_workersOverrides the default worker process count
    image_build_compute_poolBuilds the image on a cheaper pool instead of the serving pool

    GPU requests fail in two predictable ways. If the model is a type that only runs on CPU, such as scikit-learn, asking for GPUs makes the image build fail. And a model built with the Snowpark ML modeling classes can't be deployed to a GPU environment at all. The workaround is to extract the underlying native model, for example the plain XGBoost estimator, and deploy that. Table functions are also unsupported, so a model whose method is a table function can't be served.

    Expect the first deployment to take time: up to 10 minutes for a CPU model and 20 for a GPU model, and longer if the pool is idle or resizing. Calling create_service() again does not rebuild the image each time. Snowflake does rebuild it automatically when it updates the inference service, for example to patch a vulnerable dependency.

    Checkpoint 2 of 8· Check yourself

    A team trained a fraud model with the Snowpark ML modeling class XGBRegressor. They registered it and now want to serve it from a GPU compute pool through create_service. What should they do?

    Checkpoint 3 of 8· Exam question

    A retail mobile app must obtain fraud scores from a registered model in well under a second per call, from an external HTTPS client, with traffic that scales up and down through the day. Which deployment pattern fits this requirement?

    Sources3

    3.Finding and calling the inference endpoint

    Each service has an internal DNS name. Deploying with ingress_enabled=True also creates a public HTTPS endpoint. To find the public one, run SHOW ENDPOINTS and read the ingress_url column, which has the form unique-service-id-account-id.snowflakecomputing.app. Private link users read privatelink_ingress_url instead. For calls from inside Snowflake, DESCRIBE SERVICE returns the dns_name, and SHOW ENDPOINTS IN SERVICE returns the port, which gives http://dns_name:port. The endpoint is always named inference on port 5000, and neither can be changed. In Python, mv.list_services() returns both addresses as inference_endpoint and internal_endpoint.

    Each model method has its own URL path, and the name is changed in one way: underscores become dashes. A model's predict_proba method is therefore called at https://<ingress_url>/predict-proba, not /predict_proba. The path is not stripped and not split on the underscore.

    Calling the predict method with a Programmatic Access Token and a dataframe_records bodybash
    curl -X POST "<endpoint_url>/predict" \
      -H 'Authorization: Snowflake Token="<pat_token>"' \
      -H 'Content-Type: application/json' \
      -d '{
            "dataframe_records": [
                {"input_text": "Hello, world!"}
            ],
            "params": {"temperature": 0.9, "max_tokens": 512}
          }'

    The body can use either dataframe_records (one object per row) or dataframe_split (columns and data arrays). Generate it with pandas, using df.to_json(orient="split") or orient="records", then pass the result through json.loads. That way timestamps, nulls and floats are serialised the same way every time. An optional params key overrides signature parameters such as temperature.

    A 404 doesn't always mean the URL is wrong. For security, Snowflake returns 404 Not Found for every authorization failure, including an invalid token or a missing network route, and the caller can't tell these apart from a bad path. When a RUNNING service answers every PAT-authenticated request with 404, check the token and the network route before you look at the payload or the signature.

    Checkpoint 4 of 8· Check yourself

    Your service shows status RUNNING. Every POST you send from an external client to https://<ingress_url>/predict with a PAT in the Authorization header comes back 404 Not Found. What should you check first?

    Checkpoint 5 of 8· Exam question

    A team deployed a model service with a public endpoint and calls it with a Programmatic Access Token. Every request returns HTTP 404, even though the service shows as RUNNING. What is the most likely class of cause?

    Checkpoint 6 of 8· Exam question

    A model exposes a method named predict_proba. Which URL path segment should a REST client use to call that method on the service endpoint?

    Sources34

    4.Sizing and scaling the service

    min_instances and max_instances in create_service set the range the service scales within. The service starts with min_instances and scales up or down with traffic and hardware use. A scaling trigger fires after about a minute of meeting its condition, and provisioning new instances adds more time on top. For production, set min_instances to 1 or more to avoid cold starts. Set max_instances to cover peak demand while capping cost. You can also suspend or resume a service yourself with ALTER SERVICE ... SUSPEND | RESUME.

    Scale out by adding instances, not by picking bigger nodes. By default, one inference server instance claims the whole node, so max_instances only helps if the compute pool has enough nodes to hold that many instances. Choose a CPU or GPU instance family that matches the model, then set the pool's node count to at least the instance ceiling. For GPU models, use the smallest node that fits the model in memory. If the model fits on GPU_NV_S, use gpu_requests=1 and raise max_instances. If the smallest node has 4 GPUs and the model needs 2, set num_workers=2.

    Worker count is the usual cause of out-of-memory failures. A CPU model gets twice the CPU count plus one worker processes by default, and a GPU model gets one. Some models are not thread-safe, so the service loads a separate copy of the model for each worker. With a large model and many workers, that runs out of memory. Lower num_workers, or use cpu_requests, memory_requests and gpu_requests to size each instance explicitly.

    Resilience is your job, not the platform's. To survive node failures, over-provision by 50% or keep at least 3 instances, whichever is higher, so a service that needs 4 instances at peak should get 6. To survive the loss of a whole zone, run the service on a compute pool with PLACEMENT_GROUP = 'DISTRIBUTED'. This setting can only be changed while the pool is suspended.

    Checkpoint 7 of 8· Check yourself

    Instances of a large scikit-learn model, deployed with create_service using default settings, keep failing with out-of-memory errors as soon as they load. Which change addresses the cause?

    Sources536

    5.Reading latency, throughput and request metrics for a real-time endpoint

    Each service behind a REST endpoint emits performance and health metrics: request rates, latency and resource use for every instance. They go to the account's event table, so you can analyse a service's behaviour over time, not only while you are watching. The model server's container is always named model-inference. Filter on it so the image build container, model-build, doesn't mix into the results.

    Pulling the last hour of metrics for one instance with the service helper functionsql
    -- Retrieve metrics using the service helper function
    SELECT *
    FROM TABLE(mydb.myschema.my_model_service!SPCS_GET_METRICS())
    WHERE
    timestamp > dateadd(hour, -1, current_timestamp())
    AND instance_id = 0  -- choose all instances or one particular
    AND container_name = 'model-inference';

    There are three ways to read the same data. SPCS_GET_METRICS() works for running and suspended services. You can also query the event table directly with RECORD_TYPE = 'METRIC', filter on RESOURCE_ATTRIBUTES:"snow.service.name" and read the metric name from RECORD:metric:"name", which suits dashboards or joins with other telemetry. Or go to Snowsight, open Monitoring » Services & jobs, choose the service, and use its Metrics tab, filtered by instance and container.

    Logs follow the same pattern. SPCS_GET_LOGS() returns logs from the event table and needs at least MONITOR on the service. SYSTEM$GET_SERVICE_LOGS('model_service_name', '0', 'model-inference', 10) tails an active container in real time. If a latency spike on the metrics lines up with log errors from one instance_id, the problem is that instance, not the whole service.

    Checkpoint 8 of 8· Fill the gap

    This query should return the last hour of metrics from the model server only, leaving out the image build container. Which container name goes in the blank?

    -- Retrieve metrics using the service helper function
    SELECT *
    FROM TABLE(mydb.myschema.my_model_service!SPCS_GET_METRICS())
    WHERE
    timestamp > dateadd(hour, -1, current_timestamp())
    AND instance_id = 0  -- choose all instances or one particular
    AND container_name = ' ? ';

    Sources5

    Sources

    (6)

    Every claim above is drawn from one of these pages, quoted as it was written on the date shown.

    1. 1.
      “Real-time Inference (REST API): Designed for low-latency and real-time use cases.”
      ↩︎ Three inference patterns across two compute engines
      “The Snowflake Model Registry provides a unified interface to both engines.”
      ↩︎ Key concept
      “Data Source | Small inputs passed via HTTP payload. | Data residing in Snowflake Tables. | Data residing in Snowflake Stages (Files).”
      ↩︎ Checkpoint
    2. 2.
      “To host your model on a warehouse, specify WAREHOUSE in the target_platforms argument to log your model.”
      ↩︎ Three inference patterns across two compute engines
      “If the runnable_in column has WAREHOUSE as a value, you can run it.”
      ↩︎ Three inference patterns across two compute engines
      “Run an inference job on either a warehouse or an SPCS compute pool.”
      ↩︎ Three inference patterns across two compute engines
      “Snowpark DataFrames undergo lazy evaluation.”
      ↩︎ Three inference patterns across two compute engines
      “Model size exceeds 15GB. (this limit is lower for smaller warehouse sizes).”
      ↩︎ Three inference patterns across two compute engines
    3. 3.
      “ingress_enabled: This is required to be True to call online inference from outside of Snowflake.”
      ↩︎ Deploying a model version as an HTTPS service
      “the image build fails if you request GPUs”
      ↩︎ Deploying a model version as an HTTPS service
      “Calling create_service() multiple times does not trigger a build every time you call it.”
      ↩︎ Deploying a model version as an HTTPS service
      “In a URL, underscores (_) in the method name are replaced by dashes (-) in the URL.”
      ↩︎ Finding and calling the inference endpoint
      “The output contains an ingress_url column”
      ↩︎ Finding and calling the inference endpoint
      “By default, one inference server instance requests the whole node by requesting all the CPU and memory of the node it runs on.”
      ↩︎ Sizing and scaling the service
      “As a workaround, you can extract the native model and deploy that.”
      ↩︎ Exam trap 1
      “Models developed using Snowpark ML modeling classes can’t be deployed to environments that have a GPU. As a workaround, you can extract the native model”
      ↩︎ Checkpoint
      “the service loads a separate copy of the model for each worker process. This can result in resource depletion for large models.”
      ↩︎ Checkpoint
    4. 4.
      “There is currently no way to distinguish authentication errors from invalid URLs.”
      ↩︎ Finding and calling the inference endpoint
      “Snowflake returns a 404 Not Found for all authorization failures (e.g., incorrect token or lack of network route).”
      ↩︎ Exam trap 2
      “For security, Snowflake returns a 404 Not Found for all authorization failures (e.g., incorrect token or lack of network route).”
      ↩︎ Checkpoint
    5. 5.
      “Set to 1 or more for production workloads to ensure immediate availability and avoid cold-start delays.”
      ↩︎ Sizing and scaling the service
      “Snowflake recommends over-provisioning by 50% or maintaining a minimum of 3 instances (whichever is higher).”
      ↩︎ Sizing and scaling the service
      “Model serving services emit performance and health metrics that can help you monitor resource utilization, request rates, latency, and other operational characteristics.”
      ↩︎ Reading latency, throughput and request metrics for a real-time endpoint
      “The Logs, Metrics, and Events tabs provide logs, performance metrics, and service events”
      ↩︎ Reading latency, throughput and request metrics for a real-time endpoint
      “If min_instances is set to 0 (the default), the service will automatically suspend if no traffic is detected for 30 minutes.”
      ↩︎ Exam trap 3
      “If min_instances is set to 0 (the default), the service will automatically suspend if no traffic is detected for 30 minutes.”
      ↩︎ Prediction
    6. 6.
      “This can cause out-of-memory conditions for large models with a higher number of workers. Try reducing num_workers.”
      ↩︎ Sizing and scaling the service

    Continue to part 2 of 2

    Batch inference with the Model Registry: warehouses, service functions and run_batch jobs

    Spotted a mistake, or was something unclear? Tell us.