What you will be able to do
- Choose between warehouse batch, real-time SPCS service and SPCS job inference for a workload, and explain how target_platforms limits where a model can run
- Run registry-backed batch inference on a dedicated warehouse from Python and SQL, and size the warehouse for the model
- Deploy a registered model version as an HTTPS inference service with create_service and pick its compute pool, GPU and ingress settings
- Find a service's public and internal endpoints, build the URL for a model method and call it with a PAT
- Set min_instances, max_instances and num_workers so a service scales without cold starts or out-of-memory failures
- Read latency, request-rate and resource metrics for a real-time inference service
Key concept
Two engines, three inference patterns — Every registered model runs on one of two engines: a virtual warehouse (the SQL engine) or Snowpark Container Services. On top of them sit three patterns: SQL batch inference on a warehouse, a long-running SPCS service with an HTTPS endpoint, and a one-off SPCS batch job. Which one you pick depends on latency, data type and scale.
1.Three inference patterns across two compute engines
Inference in Snowflake runs on one of two compute engines: the virtual warehouse, which is the SQL engine, or Snowpark Container Services (SPCS). The Model Registry gives you one interface to both, so the same logged model version can be scored from SQL, from Python or over HTTP. The real decision is which of three patterns fits the workload.
| Aspect | Real-time inference (SPCS) | Native batch inference (SQL) | Job-based batch (SPCS) |
|---|---|---|---|
| Primary goal | Low-latency, sub-second responses for live users | Embedding models in SQL data pipelines | Large-scale, decoupled compute for unstructured data |
| Best for | Web and mobile app backends; high-concurrency request spikes | Dynamic Tables, Snowpark, dbt; SQL-first analysts and data engineers | Images, video and audio files; large historical backfills; multimodal data |
| Data source | Small inputs passed in the HTTP payload | Data in Snowflake tables | Data in Snowflake stages (files) |
| Scaling | Horizontal autoscaling to match request volume | Serverless scaling through virtual warehouses | High-throughput distributed processing of bulk data |
The latency requirement usually settles the choice. A mobile app that needs a fraud score in well under a second, on traffic that rises and falls during the day, needs a real-time service: a model hosted on an SPCS compute pool behind an HTTPS endpoint that autoscales horizontally. Running MODEL(...)!predict in a SQL statement for each request puts a warehouse query on every call. Starting a run_batch job per request means building an image, provisioning a job and writing Parquet to a stage before the caller gets anything back. Both are batch patterns, and neither fits an interactive call path.
Where a model is allowed to run is fixed when you log it. The target_platforms argument of log_model lists the engines the model is prepared for, and you must include WAREHOUSE there before the model can run inside a warehouse. For a model that is already logged, run SHOW VERSIONS IN MODEL and look at the runnable_in column. A warehouse can't host a model that needs a GPU, and it struggles with large models. Those models go to SPCS, where the same version can serve as an online service or run as a batch job.
Registry-backed batch execution on a warehouse runs a model version fetched from the Model Registry, so the model never leaves Snowflake and the data never leaves the warehouse. From Python, mv.run(input_features, function_name="predict") on a ModelVersion scores a Snowpark or pandas DataFrame and returns the same DataFrame type it was given. A Snowpark result is lazy: nothing executes until collect, show or to_pandas is called. In SQL, SELECT MODEL(my_model,LAST)!predict(...) FROM my_table does the same over a table, and the alias or version in the call decides which registered version scores the rows.
The warehouse that runs the call is the capacity you are spending, so a large scoring pass belongs on a warehouse of its own, sized for the model. Model size is the limit to watch: a model above 15GB can't run on a warehouse, and the ceiling is lower on smaller sizes. A scikit-learn or XGBoost model scoring millions of table rows fits comfortably. A model that needs a GPU, or a heavy deep-learning model, has to go to SPCS instead.
Checkpoint 1 of 8· Match them up
Match each inference pattern to the workload it fits best
Tap a term, then the definition that fits it.
Latency and data source decide the pattern. Small HTTP payloads that need sub-second answers go to a real-time SPCS service. Rows in tables scored inside SQL pipelines go to native batch inference. Bulk files in stages go to a job-based SPCS batch run.
“Data Source | Small inputs passed via HTTP payload. | Data residing in Snowflake Tables. | Data residing in Snowflake Stages (Files).”Source: docs.snowflake.com
2.Deploying a model version as an HTTPS service
create_service on a ModelVersion object turns a registered model into a managed HTTP server on SPCS. Snowflake builds the container image and runs it on the compute pool you name, so you don't manage Docker images or Kubernetes. To deploy, you need USAGE or OWNERSHIP on the compute pool, OWNER or READ on the model, and the account-level BIND SERVICE ENDPOINT privilege if the service will have a public endpoint.
# reg is a snowflake.ml.registry.Registry object
example_mv_object = reg.get_model("mymodel_name").version("version_name") # a snowflake.ml.model.ModelVersion object
example_mv_object.create_service(service_name="myservice",
service_compute_pool="my_compute_pool",
ingress_enabled=True,
gpu_requests=None)| Argument | Effect |
|---|---|
| service_name | Name of the service; must be unique in the account |
| service_compute_pool | Existing pool that runs the model; SYSTEM_COMPUTE_POOL_CPU or SYSTEM_COMPUTE_POOL_GPU can be used if the model fits |
| ingress_enabled | Must be True to call the service from outside Snowflake |
| gpu_requests | Number of GPUs; decides CPU or GPU execution for models that support both |
| num_workers | Overrides the default worker process count |
| image_build_compute_pool | Builds the image on a cheaper pool instead of the serving pool |
GPU requests fail in two predictable ways. If the model is a type that only runs on CPU, such as scikit-learn, asking for GPUs makes the image build fail. And a model built with the Snowpark ML modeling classes can't be deployed to a GPU environment at all. The workaround is to extract the underlying native model, for example the plain XGBoost estimator, and deploy that. Table functions are also unsupported, so a model whose method is a table function can't be served.
Expect the first deployment to take time: up to 10 minutes for a CPU model and 20 for a GPU model, and longer if the pool is idle or resizing. Calling create_service() again does not rebuild the image each time. Snowflake does rebuild it automatically when it updates the inference service, for example to patch a vulnerable dependency.
Checkpoint 2 of 8· Check yourself
A team trained a fraud model with the Snowpark ML modeling class XGBRegressor. They registered it and now want to serve it from a GPU compute pool through create_service. What should they do?
A model built with the Snowpark ML modeling classes can't be deployed to a GPU environment at all, whatever gpu_requests or pool you choose. The documented workaround is to extract the native model, here the plain XGBoost estimator, and deploy that.
“Models developed using Snowpark ML modeling classes can’t be deployed to environments that have a GPU. As a workaround, you can extract the native model”Source: docs.snowflake.com
Checkpoint 3 of 8· Exam question
A retail mobile app must obtain fraud scores from a registered model in well under a second per call, from an external HTTPS client, with traffic that scales up and down through the day. Which deployment pattern fits this requirement?
Correct answer: C — Create a model service on an SPCS compute pool with ingress_enabled=True and call its HTTPS endpoint from the backend.
- A. Incorrect: routing every prediction through a warehouse SQL API adds query compilation and warehouse dependency, and does not provide a managed model endpoint.
- B. Incorrect: warehouse SQL inference suits batch scoring integrated with SQL pipelines; per-request statements add query overhead and are not an external HTTPS inference API.
- C. Correct: a model service on SPCS exposes a managed HTTPS REST endpoint designed for low-latency, real-time inference and scales horizontally with its compute pool.
- D. Incorrect: run_batch builds an image and starts an SPCS job per invocation, which has startup latency measured in minutes and is meant for high-volume stages.
Sources3
3.Finding and calling the inference endpoint
Each service has an internal DNS name. Deploying with ingress_enabled=True also creates a public HTTPS endpoint. To find the public one, run SHOW ENDPOINTS and read the ingress_url column, which has the form unique-service-id-account-id.snowflakecomputing.app. Private link users read privatelink_ingress_url instead. For calls from inside Snowflake, DESCRIBE SERVICE returns the dns_name, and SHOW ENDPOINTS IN SERVICE returns the port, which gives http://dns_name:port. The endpoint is always named inference on port 5000, and neither can be changed. In Python, mv.list_services() returns both addresses as inference_endpoint and internal_endpoint.
Each model method has its own URL path, and the name is changed in one way: underscores become dashes. A model's predict_proba method is therefore called at https://<ingress_url>/predict-proba, not /predict_proba. The path is not stripped and not split on the underscore.
curl -X POST "<endpoint_url>/predict" \
-H 'Authorization: Snowflake Token="<pat_token>"' \
-H 'Content-Type: application/json' \
-d '{
"dataframe_records": [
{"input_text": "Hello, world!"}
],
"params": {"temperature": 0.9, "max_tokens": 512}
}'The body can use either dataframe_records (one object per row) or dataframe_split (columns and data arrays). Generate it with pandas, using df.to_json(orient="split") or orient="records", then pass the result through json.loads. That way timestamps, nulls and floats are serialised the same way every time. An optional params key overrides signature parameters such as temperature.
A 404 doesn't always mean the URL is wrong. For security, Snowflake returns 404 Not Found for every authorization failure, including an invalid token or a missing network route, and the caller can't tell these apart from a bad path. When a RUNNING service answers every PAT-authenticated request with 404, check the token and the network route before you look at the payload or the signature.
Checkpoint 4 of 8· Check yourself
Your service shows status RUNNING. Every POST you send from an external client to https://<ingress_url>/predict with a PAT in the Authorization header comes back 404 Not Found. What should you check first?
For security, Snowflake returns 404 for every authorization failure, such as an invalid token or a missing network route. On a running service, rule out authorization before you debug the payload or the signature.
“For security, Snowflake returns a 404 Not Found for all authorization failures (e.g., incorrect token or lack of network route).”Source: docs.snowflake.com
Checkpoint 5 of 8· Exam question
A team deployed a model service with a public endpoint and calls it with a Programmatic Access Token. Every request returns HTTP 404, even though the service shows as RUNNING. What is the most likely class of cause?
Correct answer: D — An authorization problem such as an invalid token or missing network route, because all authorization failures surface as 404.
- A. Incorrect: a signature or payload problem produces an error from the model server about the input, not a 404 for every call.
- B. Incorrect: a service reported as RUNNING is on an active pool; pool suspension is not signaled by an endpoint-wide 404.
- C. Incorrect: both dataframe_split and dataframe_records are accepted request layouts, so using one of them does not cause a 404.
- D. Correct: the documentation states that authorization failures, including an incorrect token or lack of a network route to the service, are reported as 404.
Checkpoint 6 of 8· Exam question
A model exposes a method named predict_proba. Which URL path segment should a REST client use to call that method on the service endpoint?
Correct answer: B — /predict-proba, because underscores in the method name are replaced by dashes in the endpoint URL.
- A. Incorrect: the underscore form is not used in the URL; the endpoint converts underscores to dashes.
- B. Correct: Snowflake maps underscores in method names to dashes, so predict_proba is served at predict-proba.
- C. Incorrect: the underscores are replaced rather than removed, so the joined name would not match any method.
- D. Incorrect: underscores are not treated as path separators, so a nested path would not route to the method.
4.Sizing and scaling the service
min_instances and max_instances in create_service set the range the service scales within. The service starts with min_instances and scales up or down with traffic and hardware use. A scaling trigger fires after about a minute of meeting its condition, and provisioning new instances adds more time on top. For production, set min_instances to 1 or more to avoid cold starts. Set max_instances to cover peak demand while capping cost. You can also suspend or resume a service yourself with ALTER SERVICE ... SUSPEND | RESUME.
Scale out by adding instances, not by picking bigger nodes. By default, one inference server instance claims the whole node, so max_instances only helps if the compute pool has enough nodes to hold that many instances. Choose a CPU or GPU instance family that matches the model, then set the pool's node count to at least the instance ceiling. For GPU models, use the smallest node that fits the model in memory. If the model fits on GPU_NV_S, use gpu_requests=1 and raise max_instances. If the smallest node has 4 GPUs and the model needs 2, set num_workers=2.
Worker count is the usual cause of out-of-memory failures. A CPU model gets twice the CPU count plus one worker processes by default, and a GPU model gets one. Some models are not thread-safe, so the service loads a separate copy of the model for each worker. With a large model and many workers, that runs out of memory. Lower num_workers, or use cpu_requests, memory_requests and gpu_requests to size each instance explicitly.
Resilience is your job, not the platform's. To survive node failures, over-provision by 50% or keep at least 3 instances, whichever is higher, so a service that needs 4 instances at peak should get 6. To survive the loss of a whole zone, run the service on a compute pool with PLACEMENT_GROUP = 'DISTRIBUTED'. This setting can only be changed while the pool is suspended.
Checkpoint 7 of 8· Check yourself
Instances of a large scikit-learn model, deployed with create_service using default settings, keep failing with out-of-memory errors as soon as they load. Which change addresses the cause?
Some models are not thread-safe, so the service loads a separate copy of the model for each worker. By default a CPU model gets twice the CPU count plus one workers, so lowering num_workers cuts the number of copies in memory. Adding instances doesn't help, because each new instance loads the same number of copies. A node with more vCPUs gets more workers by default, and each one loads another copy.
“the service loads a separate copy of the model for each worker process. This can result in resource depletion for large models.”Source: docs.snowflake.com
5.Reading latency, throughput and request metrics for a real-time endpoint
Each service behind a REST endpoint emits performance and health metrics: request rates, latency and resource use for every instance. They go to the account's event table, so you can analyse a service's behaviour over time, not only while you are watching. The model server's container is always named model-inference. Filter on it so the image build container, model-build, doesn't mix into the results.
-- Retrieve metrics using the service helper function
SELECT *
FROM TABLE(mydb.myschema.my_model_service!SPCS_GET_METRICS())
WHERE
timestamp > dateadd(hour, -1, current_timestamp())
AND instance_id = 0 -- choose all instances or one particular
AND container_name = 'model-inference';There are three ways to read the same data. SPCS_GET_METRICS() works for running and suspended services. You can also query the event table directly with RECORD_TYPE = 'METRIC', filter on RESOURCE_ATTRIBUTES:"snow.service.name" and read the metric name from RECORD:metric:"name", which suits dashboards or joins with other telemetry. Or go to Snowsight, open Monitoring » Services & jobs, choose the service, and use its Metrics tab, filtered by instance and container.
Logs follow the same pattern. SPCS_GET_LOGS() returns logs from the event table and needs at least MONITOR on the service. SYSTEM$GET_SERVICE_LOGS('model_service_name', '0', 'model-inference', 10) tails an active container in real time. If a latency spike on the metrics lines up with log errors from one instance_id, the problem is that instance, not the whole service.
Checkpoint 8 of 8· Fill the gap
This query should return the last hour of metrics from the model server only, leaving out the image build container. Which container name goes in the blank?
-- Retrieve metrics using the service helper function
SELECT *
FROM TABLE(mydb.myschema.my_model_service!SPCS_GET_METRICS())
WHERE
timestamp > dateadd(hour, -1, current_timestamp())
AND instance_id = 0 -- choose all instances or one particular
AND container_name = ' ? ';The model server's container is always named model-inference. model-build is the image build container, which is the one this filter keeps out. 'inference' is the name of the endpoint, not of a container.
Source: docs.snowflake.comSources5
Sources
(6)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.https://docs.snowflake.com/en/developer-guide/snowflake-ml/inference/inference-overviewOfficial docs
“Real-time Inference (REST API): Designed for low-latency and real-time use cases.”
↩︎ Three inference patterns across two compute engines“The Snowflake Model Registry provides a unified interface to both engines.”
↩︎ Key concept“Data Source | Small inputs passed via HTTP payload. | Data residing in Snowflake Tables. | Data residing in Snowflake Stages (Files).”
↩︎ Checkpoint - 2.https://docs.snowflake.com/en/developer-guide/snowflake-ml/inference/native-batch-inference-sqlOfficial docs
“To host your model on a warehouse, specify WAREHOUSE in the target_platforms argument to log your model.”
↩︎ Three inference patterns across two compute engines“If the runnable_in column has WAREHOUSE as a value, you can run it.”
↩︎ Three inference patterns across two compute engines“Run an inference job on either a warehouse or an SPCS compute pool.”
↩︎ Three inference patterns across two compute engines“Snowpark DataFrames undergo lazy evaluation.”
↩︎ Three inference patterns across two compute engines“Model size exceeds 15GB. (this limit is lower for smaller warehouse sizes).”
↩︎ Three inference patterns across two compute engines - 3.https://docs.snowflake.com/en/developer-guide/snowflake-ml/inference/real-time-inference-rest-apiOfficial docs
“ingress_enabled: This is required to be True to call online inference from outside of Snowflake.”
↩︎ Deploying a model version as an HTTPS service“the image build fails if you request GPUs”
↩︎ Deploying a model version as an HTTPS service“Calling create_service() multiple times does not trigger a build every time you call it.”
↩︎ Deploying a model version as an HTTPS service“In a URL, underscores (_) in the method name are replaced by dashes (-) in the URL.”
↩︎ Finding and calling the inference endpoint“The output contains an ingress_url column”
↩︎ Finding and calling the inference endpoint“By default, one inference server instance requests the whole node by requesting all the CPU and memory of the node it runs on.”
↩︎ Sizing and scaling the service“As a workaround, you can extract the native model and deploy that.”
↩︎ Exam trap 1“Models developed using Snowpark ML modeling classes can’t be deployed to environments that have a GPU. As a workaround, you can extract the native model”
↩︎ Checkpoint“the service loads a separate copy of the model for each worker process. This can result in resource depletion for large models.”
↩︎ Checkpoint - 4.https://docs.snowflake.com/en/developer-guide/snowflake-ml/inference/stable-endpoints-api-referenceOfficial docs
“There is currently no way to distinguish authentication errors from invalid URLs.”
↩︎ Finding and calling the inference endpoint“Snowflake returns a 404 Not Found for all authorization failures (e.g., incorrect token or lack of network route).”
↩︎ Exam trap 2“For security, Snowflake returns a 404 Not Found for all authorization failures (e.g., incorrect token or lack of network route).”
↩︎ Checkpoint - 5.https://docs.snowflake.com/en/developer-guide/snowflake-ml/inference/service-managementOfficial docs
“Set to 1 or more for production workloads to ensure immediate availability and avoid cold-start delays.”
↩︎ Sizing and scaling the service“Snowflake recommends over-provisioning by 50% or maintaining a minimum of 3 instances (whichever is higher).”
↩︎ Sizing and scaling the service“Model serving services emit performance and health metrics that can help you monitor resource utilization, request rates, latency, and other operational characteristics.”
↩︎ Reading latency, throughput and request metrics for a real-time endpoint“The Logs, Metrics, and Events tabs provide logs, performance metrics, and service events”
↩︎ Reading latency, throughput and request metrics for a real-time endpoint“If min_instances is set to 0 (the default), the service will automatically suspend if no traffic is detected for 30 minutes.”
↩︎ Exam trap 3“If min_instances is set to 0 (the default), the service will automatically suspend if no traffic is detected for 30 minutes.”
↩︎ Prediction - 6.https://docs.snowflake.com/en/developer-guide/snowflake-ml/inference/real-time-inference-troubleshootingOfficial docs
“This can cause out-of-memory conditions for large models with a higher number of workers. Try reducing num_workers.”
↩︎ Sizing and scaling the service