Run Third-Party Models in Snowflake: SPCS and Model Registry

1.Snowpark Container Services: what it is and the environment it needs

Snowpark Container Services (SPCS) is a managed container orchestration platform inside Snowflake. You package an application and its dependencies into an Open Container Initiative (OCI) image, which can contain any language, framework or library. The documentation names custom runtimes, specialized libraries and GPU workloads such as ML model serving and training as the use cases. Snowflake runs the infrastructure, and you control what goes in the container. Your container can still connect to Snowflake, run SQL in a virtual warehouse, read files from a stage, and use your existing network policies, external access integrations, role-based access control and event tables.

On top of the usual databases and warehouses, SPCS adds three objects: an image repository that stores your images, a compute pool of VM nodes that runs them, and a service that defines what runs. A workload runs in one of two forms:

Environment setup. The common tutorial setup lists these prerequisites: a Snowflake account (trial accounts are not supported), a SQL client such as SnowSQL or Snowsight, and Docker Desktop. Any OCI-compliant client, such as Podman or Nerdctl, also works for building images. The setup script runs as ACCOUNTADMIN. It creates a dedicated role, gives that role ownership of a database, creates an X-SMALL warehouse and grants USAGE on it, grants BIND SERVICE ENDPOINT on the account (needed for a service that exposes a public endpoint), creates a compute pool and grants USAGE and MONITOR on it, and then grants the role to the user who will do the work.

The warehouse can look out of place, because containers run on the compute pool and not in a warehouse. It is there because services, including job services, can run SQL DML statements, and Snowflake runs those statements in a warehouse. SQL is the main interface in the documentation, but Python APIs, REST APIs and the Snowflake CLI cover most of the same operations.

Sources12

2.Image repositories and Docker images

Snowflake runs an OCIv2-compliant image registry for each account. Inside it you create repositories, which are named storage units for images. The documentation compares the two to a DBMS and a table: the registry is the DBMS and each repository is a table. You create a repository with CREATE IMAGE REPOSITORY, which needs the CREATE IMAGE REPOSITORY privilege on the schema. You manage repositories with DROP IMAGE REPOSITORY and SHOW IMAGE REPOSITORIES, and list their contents with SHOW IMAGES IN IMAGE REPOSITORY. You might create separate DEV, TEST and PROD repositories, or give different repositories different permissions.

The registry hostname follows the pattern <orgname>-<acctname>.registry.snowflakecomputing.com, and a repository URL is <registry-hostname>/<db_name>/<schema_name>/<repository_name>. URLs can't contain underscores, so an account name like my_account becomes my-account. SHOW IMAGE REPOSITORIES returns the repository URL for you.

You have to authenticate before Docker can push. One option is the Snowflake CLI command snow spcs image-registry login. Another is docker login with a programmatic access token (PAT): the username is "USER" and the token is the password. Plain username/password login works only if an administrator enables it, because MFA is incompatible with docker login.

Docker images. You build the image from a Dockerfile for the linux/amd64 platform. Then you tag it with the repository URL and push it. In the containers tutorial the build command looks like this:

docker build --rm --platform linux/amd64 -t my_echo_service_image:tutorial .

Next you get the repository URL with snow spcs image-repository url or SHOW IMAGE REPOSITORIES, and tag the image with docker tag <image_name> <image_url>/<image_name> so that the push lands in your Snowflake repository. Access control applies to the whole repository. You can't set permissions on individual images. You also can't drop a single image: dropping the repository removes all of its images. The largest compressed layer allowed is 160 GiB on AWS and 195 GiB on Azure.

Sources3

3.Creating a compute pool

Services and job services run on a compute pool, a collection of one or more VM nodes. A compute pool is an account-level object, much like a virtual warehouse, so each pool name must be unique in the account. You create the pool first and then name it when you create a service or job service. You need three pieces of information to create one:

- Instance family: the machine type. Choosing it is like choosing a warehouse size, and some families provide GPUs. SHOW COMPUTE POOL INSTANCE FAMILIES lists the available families. - Minimum nodes: how many nodes the pool launches with. - Maximum nodes: the upper limit for Snowflake's autoscaling.

You can create a pool with SQL, or in Snowsight under Compute » Compute Pools using ACCOUNTADMIN or another role that is allowed to create pools.

CREATE COMPUTE POOL tutorial_compute_pool MIN_NODES = 1 MAX_NODES = 1 INSTANCE_FAMILY = CPU_X64_XS;

After creation, Snowflake launches the minimum number of nodes and adds more, up to the maximum, when the running nodes can't take any more work. It also removes nodes that have run no services for a while, but never goes below the minimum. Setting the minimum above 1 keeps spare capacity ready for bursts of load instead of waiting for autoscaling. Setting a maximum caps cost if a load spike or a bug would otherwise make Snowflake add many nodes.

The placement_group setting is optional. Without it, Snowflake places nodes wherever capacity is available. With DISTRIBUTED, Snowflake tries to spread nodes across placement groups for fault tolerance. If a placement group fails, Snowflake doesn't automatically fail nodes over to a healthy group, so the documentation recommends overprovisioning service instances (N+1).

Sources4

4.Service specification files

The pool supplies the machines and the repository holds the image. The specification file, written in YAML and supplied when you create the service, tells Snowflake how to run the image. It has two top-level fields: spec and serviceRoles. Service roles control privileges on the endpoints the service exposes. Under spec, only containers is required. A service must have at least one container, and each container needs only a name and an image, where the image is one you uploaded to a repository in your account. All containers of one service instance run on a single node and share its network interface. Container, endpoint and volume names can be up to 63 characters long, may contain only lowercase alphanumerics and -, must start with a letter and must end with an alphanumeric character.

The spec can override what you put in the Dockerfile, so changing a container's behaviour doesn't require rebuilding the image. containers.command overrides the Dockerfile ENTRYPOINT, and containers.args overrides CMD. containers.env sets environment variables for every process in the container. The example below sets two variables. Because one of them changes the listening port, the endpoint's port has to change to match:

spec:
  containers:
  - name: echo
    image: <image_name>
    env:
      CHARACTER_NAME: Bob
      SERVER_PORT: 8085
  endpoints:
  - name: echo-endpoint
    port: 8085

Other container fields you'll see:

- readinessProbe: Snowflake sends an HTTP GET to the given port and path, expects HTTP 200 OK, and stops routing traffic to the container while the probe fails. - resources: requests and limits for memory, CPU and nvidia.com/gpu. - secrets: points to a snowflakeSecret by objectName or objectReference, and delivers it to the container through an envVarName or a directoryPath. The credential then never has to appear in the image or the spec.

Under the top-level capabilities.securityContext, executeAsCaller signals that the application intends to use caller's rights. Don't put personal, sensitive or regulated data in the specification as metadata.

Sources5

5.Model Registry: logging the model

SPCS means building and running containers yourself. The Snowflake Model Registry is the shorter route for a trained Python model. It stores models as schema-level objects together with their versions, metrics and metadata, and runs inference from Python, SQL or REST endpoints. Its built-in types include scikit-learn, xgboost, LightGBM, Prophet, CatBoost, PyTorch, TensorFlow, Keras, Hugging Face pipelines, Sentence Transformer and MLFlow pyfunc models, and it also accepts custom models. Models trained with Snowflake ML Functions such as FORECAST don't appear in the registry. The API is generally available from snowflake-ml-python 1.5.0.

Any schema can act as a registry without setup, though Snowflake recommends a dedicated schema such as ML.REGISTRY. You start by opening the registry:

from snowflake.ml.registry import Registry

reg = Registry(session=session, database_name="ML", schema_name="REGISTRY")

Adding a model is called logging it. log_model serializes the Python model object, creates a Snowflake model object from it, and records metadata such as a comment and metrics. It has two required arguments: model, which must be picklable, and model_name, which can't be changed later. The model name and version name together must be unique in the schema. To add a version, call log_model again with the same model_name and a new version_name. conda_dependencies lists packages to deploy with the model. You can't add tags in the log_model call, because the model object is only created when its first version is logged. Add tags afterwards.

To create a model you must own the schema or hold CREATE MODEL on it. The limits are up to 1000 versions per model, 10 methods per version, and 15 GB total model size for warehouse-deployed models. You can also import models from an external provider.

mv = reg. ? (clf,
                   model_name="my_model",
                   version_name="v1",
                   conda_dependencies=["scikit-learn"],
                   comment="My awesome ML model",
                   metrics={"score": 96},
                   sample_input_data=train_features,
                   task=task.Task.TABULAR_BINARY_CLASSIFICATION)

Sources6

6.Model Registry: calling the model

A logged model version has methods, such as predict, which are functions attached to the model. Different versions of a model can have different methods with different signatures. In Python, you call a method with mv.run, where mv is a ModelVersion. You pass a Snowpark or pandas DataFrame and the function_name, and you get back a DataFrame of the same type. The method runs in the warehouse of your current session. Snowpark DataFrames are evaluated lazily, so nothing runs until you call collect, show or to_pandas.

remote_prediction = mv.run(test_features, function_name="predict")

In SQL, the syntax is MODEL(<model_name>)!<method_name>(...), with the inference table in the FROM clause. If you don't name a version, the call goes to the model's default version, not automatically to the newest one. To target a specific version, pass a version name or alias as the second argument. The built-in LAST alias selects the latest version. SHOW FUNCTION IN MODEL ... VERSION ... lists a version's methods and their signatures.

SELECT MODEL(my_model,LAST)!predict(...) FROM my_table;

Calling a model also needs privileges. USAGE allows warehouse inference without exposing the model's internals. READ allows SPCS inference and also shows metadata such as comments, tags and metrics. Both can be granted on all current or future models in a schema, for example with GRANT USAGE ON FUTURE MODELS IN SCHEMA.

Sources67

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.SPCS accepts images built for any platform, including ARM images built on a laptop.Why is that wrong?

   SPCS currently requires linux/amd64 images, so you build with --platform linux/amd64.

   Covered in Image repositories and Docker images

2. 2.You can delete one outdated image from a repository, or restrict access to individual images.Why is that wrong?

   Dropping individual images isn't supported, and access control applies to the whole repository. Dropping the repository removes all of its images.

   Covered in Image repositories and Docker images

3. 3.Compute pools are schema-level objects, so two schemas can each have a pool with the same name.Why is that wrong?

   A compute pool is an account-level object, like a warehouse, so its name must be unique in the account.

   Covered in Creating a compute pool

4. 4.You can attach tags to a model in the log_model call.Why is that wrong?

   log_model adds a version and creates the model only when its first version is logged. Tags are attributes of the model, so you add them afterwards.

   Covered in Model Registry: logging the model

5. 5.USAGE on a model is enough for every kind of inference, including models served in SPCS.Why is that wrong?

   USAGE covers warehouse inference only. SPCS inference requires READ, which also shows the model's metadata.

   Covered in Model Registry: calling the model