What you will be able to do
- Run registry-backed batch inference in SQL and Python on a virtual warehouse
- Route batch SQL through an existing SPCS model service as a service function, and size the calling warehouse correctly
- Launch, monitor and read the output of a run_batch job on SPCS without creating a service
- Feed images, audio and other staged files to a multimodal model through run_batch
- Tune run_batch resources and run it as a blocking step in a task DAG
1.Batch inference on a virtual warehouse
A model can be scored in SQL on a warehouse only if it was logged with WAREHOUSE among its target platforms. To check an existing version, run SHOW VERSIONS IN MODEL and look for WAREHOUSE in the runnable_in column. A version that qualifies behaves like a SQL function, so queries, Dynamic Tables, Snowpark and dbt can call it without moving data or managing a compute pool.
In SQL, call a method with MODEL(<model_name>)!<method_name>(...), which uses the default version. To pin a version, add a version name or alias, as in MODEL(my_model, LAST)!predict(...). Aliases are managed in the Model Registry. If the signature declares parameters, you pass them after the input columns, either all positionally or all by name. Mixing the two styles in one call is an error. In Python, mv.run(df, function_name="predict") runs the same inference on a warehouse and returns the same DataFrame type it was given, so pandas in gives pandas out.
CREATE OR REPLACE DYNAMIC TABLE logins_with_predictions
WAREHOUSE = my_wh
TARGET_LAG = '20 minutes'
REFRESH_MODE = INCREMENTAL
INITIALIZE = on_create
COMMENT = 'Dynamic table with continuously updated model predictions'
AS
SELECT
login_id,
user_id,
location,
event_time,
MODEL(ml.registry.mymodel)!predict(l.user_id, l.location) AS prediction_result
FROM logins_raw;Two limits rule a model out of the warehouse: it needs a GPU, or it is larger than 15GB (smaller warehouse sizes have a lower ceiling). CPU-runnable models such as scikit-learn and XGBoost fit comfortably.
Checkpoint 1 of 6· Fill the gap
This Dynamic Table scores new login rows with a registered model on a warehouse. Which keyword completes the call to the model?
CREATE OR REPLACE DYNAMIC TABLE logins_with_predictions
WAREHOUSE = my_wh
TARGET_LAG = '20 minutes'
REFRESH_MODE = INCREMENTAL
INITIALIZE = on_create
COMMENT = 'Dynamic table with continuously updated model predictions'
AS
SELECT
login_id,
user_id,
location,
event_time,
? (ml.registry.mymodel)!predict(l.user_id, l.location) AS prediction_result
FROM logins_raw;A registry model that can run in a warehouse is called from SQL as MODEL(<model_name>)!<method_name>(...). With no version or alias given, this uses the default version. It is not called as a service, a plain function or a registry object.
Source: docs.snowflake.comCheckpoint 2 of 6· Check yourself
A data engineering team wants to add model scoring to their SQL pipelines with MODEL(...)!predict on an existing warehouse. Which of these models should go to SPCS instead?
Being able to run on a CPU is not enough. A warehouse is the wrong host for any model larger than 15GB, and the limit is lower on smaller warehouse sizes. A model that needs a GPU also goes to SPCS. The other three are small CPU models, which is what warehouse batch inference is for.
“Hardware Constraints: The model requires a GPU for execution. • Memory Limits: Model size exceeds 15GB.”Source: docs.snowflake.com
Sources1
2.Batch through an online service's service function
A model deployed as an SPCS service, which is a long-running container that also serves an HTTPS endpoint, can take batch SQL as well. The same deployment then handles both online and batch traffic, with GPUs and custom packages a warehouse can't provide. The call syntax drops MODEL(...) and names the service directly. In Python, adding service_name= to mv.run sends the batch to the service instead of the warehouse.
-- Positional
SELECT my_service!predict(input_text, 0.9, 512) FROM my_table;
-- Named arguments (all arguments must be named)
SELECT my_service!predict(input_text => input_text, max_tokens => 512) FROM my_table;For a service function, the warehouse only routes requests. Inference runs on the service's compute pool. The warehouse runs many threads per node and packs many rows into each request, so a large one can swamp the service. Use XSMALL or SMALL and scale the service, not the warehouse.
Two operational details trip people up. First, only the service owner can call service functions by default, so another role needs the service role INFERENCE_SERVICE_FUNCTION_USAGE granted with GRANT SERVICE ROLE. Second, a suspended service resumes automatically when a request arrives. If the compute pool has no free nodes, the resume can take too long and the query fails. Before a critical batch run, resume the service explicitly and wait until it is available.
Checkpoint 3 of 6· Check yourself
An analytics role runs SELECT fraud_svc!predict(...) FROM transactions against a model service owned by the ML team's role, and the call is rejected. What does the ML team need to grant so that the analysts can run this batch query?
By default only the service owner can call service functions. Another role needs the INFERENCE_SERVICE_FUNCTION_USAGE service role. READ on the model, BIND SERVICE ENDPOINT and compute pool USAGE are what you need to deploy or expose a service, not to call its service functions from SQL.
“To let another role call the service (for SQL calls using service_name!method_name(...) syntax), grant the service role INFERENCE_SERVICE_FUNCTION_USAGE using GRANT SERVICE ROLE.”Source: docs.snowflake.com
Sources1
3.Job-based batch inference with run_batch
ModelVersion.run_batch (snowflake-ml-python 2.0.0 or later) runs batch inference on SPCS without a service. The job builds the inference image, starts an SPCS job on your pool, writes the results to a stage and then shuts the compute down, so you don't pay for idle capacity. It is the pattern for millions or billions of rows, for image, audio and video files, and for inference as a separate asynchronous pipeline stage. In SQL, the equivalent is EXECUTE INFERENCE JOB SERVICE.
from snowflake.ml.model.batch_inference import OutputSpec, SaveMode
input_df = session.table("my_db.my_schema.my_input_table")
job = mv.run_batch(
input_df,
compute_pool="my_compute_pool",
output_spec=OutputSpec(
stage_location="@my_db.my_schema.my_stage/predictions/",
mode=SaveMode.ERROR,
),
)
job.wait() # optional: block until the job finishesPass exactly one input source. That is either X, a Snowpark DataFrame, which Snowflake first writes as Parquet under the output location, or input_stage_location, which points at Parquet already in a stage and reads it in place. Passing both or neither raises a ValueError. The input path must not sit inside the output path, or the job could read its own output. The output stage must be internal.
The call returns an MLJob handle right away, because async_=True is the default. Use job.get_logs(), job.cancel(), list_jobs(), get_job() and delete_job() to manage it. The job's result() function is not supported here, so read the predictions from the output stage. A job that fails partway can leave partial files behind. Wait for the _SUCCESS file, start from an empty directory, and keep SaveMode.ERROR, the default, so the job fails rather than overwriting earlier output.
To make inference one node in a scheduled task graph, so that downstream tasks start only when scoring is done, use BatchInferenceTask inside a DAG. It runs synchronously, so the DAG step completes when the job does. It takes a query or an input_stage_location, not a DataFrame. For a plain script, async_=False or job.wait() gives the same blocking behaviour. Building the DAG itself belongs to workflow orchestration.
Checkpoint 4 of 6· Fill the gap
A team needs to score a billion-row table on SPCS without leaving a long-running service up afterwards. Which ModelVersion method fills the blank so the compute shuts down when scoring finishes?
from snowflake.ml.model.batch_inference import OutputSpec, SaveMode
input_df = session.table("my_db.my_schema.my_input_table")
job = mv. ? (
input_df,
compute_pool="my_compute_pool",
output_spec=OutputSpec(
stage_location="@my_db.my_schema.my_stage/predictions/",
mode=SaveMode.ERROR,
),
)
job.wait() # optional: block until the job finishesrun_batch starts a job-based batch inference run on SPCS. It builds the image, runs the job on the named pool, writes predictions to the output stage and then shuts the compute down. It returns an MLJob handle, which is why job.wait() follows. mv.run returns a DataFrame from a warehouse or an existing service. create_service deploys a long-running service, which is the thing this team wants to avoid.
Source: docs.snowflake.comSources2
4.Unstructured and multimodal data in batch jobs
To score files instead of rows, the input DataFrame holds fully qualified stage paths, one file per row, and the job reads each file and hands its content to the model. list_stage_files(session, "@db.schema.stage/path", pattern=".*\\.jpg") builds that DataFrame from a stage listing. InputSpec.column_handling tells the job which column holds paths and how to encode the file contents for the model: FileEncoding.RAW_BYTES, BASE64 or BASE64_DATA_URL.
job = mv.run_batch(
X,
compute_pool="my_compute_pool",
output_spec=OutputSpec(stage_location="@my_db.my_schema.my_stage/predictions/"),
input_spec=InputSpec(
# FULL_STAGE_PATH: the column holds a fully qualified path (@db.schema.stage/path) to a file
# RAW_BYTES: download the file and hand its bytes to the model
column_handling={
"path": ColumnHandlingOptions(
input_format=InputFormat.FULL_STAGE_PATH,
convert_to=FileEncoding.RAW_BYTES,
)
}
),
)Where the files live matters more than their format. Every type of internal stage works. External input stages must be Amazon S3 and must use server-side encryption. Azure Blob Storage and Google Cloud Storage are not supported. An S3 stage needs a one-time storage integration and IAM setup, and the job's role needs USAGE on it. One DataFrame can mix internal and external paths. Results always go to an internal stage, so 500 GB of audio in S3 can be read in place, but the transcripts land inside Snowflake.
No. For multimodal use cases only server-side encryption is supported, and S3 is the only supported external stage type. Re-encrypt the bucket server-side or copy the files to an internal stage.
Checkpoint 5 of 6· Fill the gap
In this run_batch call, the input DataFrame has a 'path' column with one staged file per row. Which InputSpec field name fills the blank, so that the job knows the column holds stage paths and what encoding to send the model?
job = mv.run_batch(
X,
compute_pool="my_compute_pool",
output_spec=OutputSpec(stage_location="@my_db.my_schema.my_stage/predictions/"),
input_spec=InputSpec(
# FULL_STAGE_PATH: the column holds a fully qualified path (@db.schema.stage/path) to a file
# RAW_BYTES: download the file and hand its bytes to the model
? ={
"path": ColumnHandlingOptions(
input_format=InputFormat.FULL_STAGE_PATH,
convert_to=FileEncoding.RAW_BYTES,
)
}
),
)InputSpec.column_handling maps each path column to a ColumnHandlingOptions entry. The entry's input_format marks the column as full stage paths, and its convert_to sets the encoding the model gets: RAW_BYTES, BASE64 or BASE64_DATA_URL. input_stage_location is a run_batch argument that points at Parquet input already in a stage. It is not an InputSpec field and does not describe file columns.
Source: docs.snowflake.comSources2
5.Tuning run_batch resources and throughput
Three optional blocks control how a job uses its pool, and each answers a different question. ResourcesSpec sets how much of a node each inference container gets. InferenceSpec sets how the model is served inside a container. replicas sets how many containers run. Leave them out and the job tries to use all of the node's vCPUs and memory, picks worker count and batch size itself, and runs on CPU.
| Setting | Block | What it controls |
|---|---|---|
| cpu_requests / memory_requests | ResourcesSpec | Per-node CPU and memory limits; omitted means use all of the node |
| gpu_requests | ResourcesSpec | GPU limit; omitted means the job runs on CPU |
| num_workers | InferenceSpec | Workers handling requests in parallel within one instance |
| max_batch_rows | InferenceSpec | Rows per batch; larger values can improve throughput |
| replicas | run_batch argument | Number of job instances running in parallel |
These settings pull in different directions. Raising max_batch_rows and num_workers speeds up a container but puts more data and work into the same memory. Adding replicas adds containers, but each one still has only one node's memory. When workers fail with out-of-memory errors on a large model, use fewer workers or smaller batches per container, or a pool with larger nodes. Recover throughput with more replicas rather than more workers. EngineOptions can also switch the serving engine, for example to InferenceEngine.VLLM for LLMs on a GPU pool.
Checkpoint 6 of 6· Check yourself
Workers in a run_batch job that scores a large model keep failing with out-of-memory errors. Which change deals with the cause instead of putting more load on the same container's memory?
Each worker loads its own copy of the model, so with a large model, many workers use up the node's memory. Larger batches add more data to that same memory. Extra replicas run more containers, but each one still has only one node's memory and fails the same way. Lower num_workers (or use smaller batches, or larger nodes), then get throughput back by adding replicas.
“the service loads a separate copy of the model for each worker process. This can result in resource depletion for large models.”Source: docs.snowflake.com
Sources2
Practise it for real
Run a registered model version as a run_batch job on SPCS and confirm its output is safe to read
1.Get a ModelVersion with registry.get_model(...).version(...) and build input_df from a feature table with session.table(...).
Why: run_batch is a method on the model version, and a Snowpark DataFrame is one of its two accepted input sources.
You should see: An mv object and a lazy DataFrame; nothing has run yet.
2.Call mv.run_batch(input_df, compute_pool=..., output_spec=OutputSpec(stage_location="@db.schema.stage/predictions/", mode=SaveMode.ERROR)) against an empty internal stage path.
Why: An empty directory plus SaveMode.ERROR guards against partial or overwritten output.
You should see: The call returns an MLJob handle immediately, because async_ defaults to True.
3.Call job.wait(), then print(job.get_logs()).
Why: wait() blocks until the job ends, and the logs show the image build and inference progress.
You should see: The job finishes and the logs show inference running on the named pool.
4.List the output stage path and look for the _SUCCESS file before reading the Parquet results.
Why: A job that fails partway can leave partial files; _SUCCESS marks a complete run.
You should see: Prediction files plus a _SUCCESS marker in the output directory.
5.Run the same run_batch call again with the same stage_location.
Why: This shows what SaveMode.ERROR does when the output directory already holds files.
You should see: The job fails instead of overwriting the earlier predictions.
Stuck? Get a nudge
If the second run overwrote your files, check that mode was SaveMode.ERROR and not SaveMode.OVERWRITE.
Sources
(3)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.https://docs.snowflake.com/en/developer-guide/snowflake-ml/inference/native-batch-inference-sqlOfficial docs
“SELECT MODEL(<model_name>,<version_or_alias_name>)!<method_name>(...) FROM <table_name>;”
↩︎ Batch inference on a virtual warehouse“You must specify all arguments by name or all by position.”
↩︎ Batch inference on a virtual warehouse“Model size exceeds 15GB.”
↩︎ Batch inference on a virtual warehouse“Unlike running in a warehouse, functions can be called from a service by calling service_name!method_name(...).”
↩︎ Batch through an online service's service function“grant the service role INFERENCE_SERVICE_FUNCTION_USAGE using GRANT SERVICE ROLE.”
↩︎ Batch through an online service's service function“Queries might fail to resume if there are a lack of available nodes in the compute pool.”
↩︎ Batch through an online service's service function“A warehouse can run multiple threads per node and send a large number of inference requests with each request.”
↩︎ Exam trap 1“Hardware Constraints: The model requires a GPU for execution. • Memory Limits: Model size exceeds 15GB.”
↩︎ Checkpoint“the recommendation is to utilize an XSMALL or SMALL warehouse when the model is deployed to SPCS.”
↩︎ Prediction“To let another role call the service (for SQL calls using service_name!method_name(...) syntax), grant the service role INFERENCE_SERVICE_FUNCTION_USAGE using GRANT SERVICE ROLE.”
↩︎ Checkpoint - 2.https://docs.snowflake.com/en/developer-guide/snowflake-ml/inference/batch-inference-jobsOfficial docs
“The job builds an inference image, provisions an SPCS job on the compute pool you name, runs inference, writes results to a stage”
↩︎ Job-based batch inference with run_batch“Passing both, or neither, raises a ValueError.”
↩︎ Job-based batch inference with run_batch“To mark completion, the job writes a _SUCCESS file to the output directory.”
↩︎ Job-based batch inference with run_batch“The job runs synchronously, so the DAG step completes when the job does.”
↩︎ Job-based batch inference with run_batch“For unstructured data, reference the files by their fully qualified stage paths in the input DataFrame.”
↩︎ Unstructured and multimodal data in batch jobs“External stages: Amazon S3 only, and the stage must use server-side encryption.”
↩︎ Unstructured and multimodal data in batch jobs“For multimodal use cases, only server-side encryption is supported.”
↩︎ Unstructured and multimodal data in batch jobs“Number of workers that handle requests in parallel within one service instance.”
↩︎ Tuning run_batch resources and throughput“If omitted, the job tries to use all memory on the node.”
↩︎ Tuning run_batch resources and throughput“Larger values can improve throughput.”
↩︎ Tuning run_batch resources and throughput“The output stage must be a Snowflake internal stage.”
↩︎ Exam trap 2“The result function in the ML Job APIs”
↩︎ Exam trap 3
Also cited
- https://docs.snowflake.com/en/developer-guide/snowflake-ml/inference/real-time-inference-rest-apiOfficial docs
“the service loads a separate copy of the model for each worker process. This can result in resource depletion for large models.”
↩︎ Checkpoint