Batch inference in Snowflake: warehouse SQL, service functions and run_batch jobs

1.Registry-backed batch inference on a warehouse

Native batch inference treats a model version in the Snowflake Model Registry like a SQL function. You can call it from SQL, Snowpark Python, streaming pipelines and Dynamic Tables, and tools such as dbt can use its results. To make a model runnable on a warehouse, include WAREHOUSE in target_platforms when you log it. To check an existing version, run SHOW VERSIONS IN MODEL and look for WAREHOUSE in the runnable_in column.

In SQL you call a method through MODEL(...)!method. Leaving out the version uses the default version, and naming a version or an alias such as LAST pins it:

SELECT MODEL(<model_name>,<version_or_alias_name>)!<method_name>(...) FROM <table_name>;

From Python, call run on the model version. The output has the same type as the input: a pandas DataFrame in gives a pandas DataFrame out. Snowpark DataFrames are evaluated lazily, so nothing executes until collect, show or to_pandas. mv.show_functions() and SHOW FUNCTION IN MODEL list the callable methods. If you put the MODEL(...)!predict call inside a Dynamic Table with REFRESH_MODE = INCREMENTAL, only newly arrived rows get scored on each refresh.

A warehouse suits CPU-runnable models such as scikit-learn or XGBoost, and it needs no compute pool. It is the wrong choice when the model needs a GPU or is larger than about 15GB. That limit is lower on smaller warehouses. Large models, GPU models and custom environments belong on SPCS instead.

# Run inference on a warehouse
# mv: snowflake.ml.model.ModelVersion
remote_prediction = mv.run(input_features, function_name="predict")
remote_prediction.show()

Sources1

2.Batch scoring through the online service as a service function

A model deployed to SPCS as a service (the same kind of service created with create_service for real-time REST calls) can also run SQL batch inference. A warehouse model is called through MODEL(...)!method. A service is called by its own name, service_name!method_name(...), and parameters work the same way: either all positional or all named.

-- Positional
SELECT my_service!predict(input_text, 0.9, 512) FROM my_table;

In Python you use the same run call as on a warehouse plus a service_name argument. Three operational rules apply:

- Access. By default only the service owner can call service functions. To let another role call it, grant that role the service role INFERENCE_SERVICE_FUNCTION_USAGE with GRANT SERVICE ROLE. - Warehouse size. The warehouse only sends requests to the compute pool. A large warehouse runs many threads and can overwhelm the service, so use XSMALL or SMALL. - Suspension. A suspended service resumes when a request arrives. If the compute pool has no free nodes, the resume can take too long and the query fails. Resume the service explicitly and wait until it is available before running the batch.

# Run inference on SPCS
# mv: snowflake.ml.model.ModelVersion
remote_prediction = mv.run(input_features, function_name="predict",  ? ="example_spcs_service")
remote_prediction.show()

Sources1

3.Job-based batch inference with run_batch, with no service

For large batch workloads you don't have to keep a service running. ModelVersion.run_batch (snowflake-ml-python 2.0.0 or later) runs inference as an SPCS job. In SQL the equivalent is EXECUTE INFERENCE JOB SERVICE. Snowflake runs the job in this order:

Use it for millions or billions of rows, for inference that runs as its own asynchronous pipeline stage, or as a step in an Airflow DAG or a Snowflake task.

from snowflake.ml.model.batch_inference import OutputSpec, SaveMode

input_df = session.table("my_db.my_schema.my_input_table")

job = mv.run_batch(
    input_df,
    compute_pool="my_compute_pool",
    output_spec=OutputSpec(
        stage_location="@my_db.my_schema.my_stage/predictions/",
        mode=SaveMode.ERROR,
    ),
)

job.wait()  # optional: block until the job finishes

run_batch is asynchronous by default (async_=True). Pass async_=False if the call should block. Give it exactly one input: a Snowpark DataFrame X (which gets materialized as Parquet under the output path) or an input_stage_location that already holds Parquet. Passing both, or neither, raises a ValueError. The input path must not sit inside the output path.

The call returns an MLJob handle, and the ML Job APIs manage it: job.get_logs(), job.cancel(), list_jobs(), get_job(), delete_job(). The result function is not supported for batch inference jobs, so read predictions from the output stage. A run that has not finished can leave output that looks truncated. Wait for the _SUCCESS file before reading. If a job fails because files already exist under SaveMode.ERROR, point it at a new directory or use SaveMode.OVERWRITE.

Sources2

4.Batch jobs over unstructured files and multimodal models

For images, audio or video, the input DataFrame holds fully qualified stage paths rather than feature values. The job reads each file and passes its contents to the model. list_stage_files(session, "@db.schema.stage/path") builds that DataFrame for you and can filter with a regex pattern and set a column_name. To tell the job which column holds the paths and how to encode the file contents, use InputSpec.column_handling. The model can receive FileEncoding.RAW_BYTES, FileEncoding.BASE64 or FileEncoding.BASE64_DATA_URL.

job = mv.run_batch(
    X,
    compute_pool="my_compute_pool",
    output_spec=OutputSpec(stage_location="@my_db.my_schema.my_stage/predictions/"),
    input_spec=InputSpec(
        # FULL_STAGE_PATH: the column holds a fully qualified path (@db.schema.stage/path) to a file
        # RAW_BYTES: download the file and hand its bytes to the model
        column_handling={
            "path": ColumnHandlingOptions(
                input_format=InputFormat.FULL_STAGE_PATH,
                convert_to=FileEncoding.RAW_BYTES,
            )
        }
    ),
)

External S3 input needs a one-time admin setup (a storage integration and IAM permissions on the bucket), and the role running the job needs USAGE on the external stage. Multimodal use cases support only server-side encryption. partition_column doesn't work with Hugging Face pipeline models or FUNCTION-type methods.

Sources2

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.A finished run_batch job hands back its predictions through job.result(), as other ML jobs do.Why is that wrong?

   result is not supported for batch inference jobs. Predictions have to be read from the output stage.

   Covered in Job-based batch inference with run_batch, with no service

2. 2.Any external stage (S3, Azure or GCS) can feed a batch inference job.Why is that wrong?

   The only supported external input is Amazon S3 with server-side encryption. Azure Blob Storage and Google Cloud Storage are not supported.

   Covered in Batch jobs over unstructured files and multimodal models

3. 3.Any role with READ on the model can call the deployed service's SQL functions.Why is that wrong?

   By default only the service owner can call service functions. Other roles need the INFERENCE_SERVICE_FUNCTION_USAGE service role, granted with GRANT SERVICE ROLE.

   Covered in Batch scoring through the online service as a service function

Practise it for real

Run a job-based batch inference over a table with no model service, and read its results safely.

1. 1.Get a ModelVersion with Registry(session=session, database_name=..., schema_name=...).get_model("my_model").version("my_version").

   Why: run_batch is a method on the model version object.

   You should see: A ModelVersion handle with no service created.

2. 2.Call mv.run_batch(input_df, compute_pool="my_compute_pool", output_spec=OutputSpec(stage_location="@my_db.my_schema.my_stage/predictions/", mode=SaveMode.ERROR)).

   Why: SaveMode.ERROR stops the job from mixing new output with files already in the directory.

   You should see: The call returns an MLJob handle at once, because async_ defaults to True.

3. 3.Call job.wait(), and if the job fails, print(job.get_logs()).

   Why: The ML Job APIs are how you inspect and troubleshoot batch jobs.

   You should see: The job completes, or the logs show why it failed.

4. 4.List the output stage path and check for the _SUCCESS file before you read predictions from it.

   Why: Output from an unfinished job can look truncated, and result() is not supported.

   You should see: A _SUCCESS file next to the prediction files.