What you will be able to do
- Create a compute pool with an instance family, node limits and auto-suspend that suit an ML workload
- Run your own container image on a compute pool, and choose a long-running service or a job service
- Decide when a Snowpark-optimized warehouse or Container Runtime should run a training workload
- Compare single-node warehouse training with the distributed trainers in Container Runtime
- Choose how a library that Snowpark doesn't provide reaches a training job: a run-time install, a custom runtime environment, or your own container image
Key concept
Compute pool — A compute pool is a set of nodes that all use one machine type. It is the hardware for every container workload in Snowflake: Snowpark Container Services, Container Runtime notebooks and ML Jobs. Its instance family sets the hardware and the credit rate, and MIN_NODES and MAX_NODES set how far it can scale.
1.Compute pools: the hardware behind Container Runtime
Every containerised ML workload in Snowflake runs on a compute pool. A virtual warehouse can't run it. Container Runtime runs inside Snowpark Container Services, which can run on CPU or GPU compute pools. Notebooks on Container Runtime, ML Jobs and plain container services all ask a pool for nodes. If you size the pool wrong, the workload is wrong before any code runs.
You set three things when you create a pool:
INSTANCE_FAMILYsets the machine type for every node. That gives the hardware (CPU, high-memory CPU or GPU) and the number of credits consumed while the pool is running.MIN_NODESis the smallest number of nodes the pool keeps, and it must be greater than 0.MAX_NODESis the largest number of nodes the pool can scale to.
CREATE COMPUTE POOL [ IF NOT EXISTS ] <name>
[ FOR APPLICATION <app-name> ]
MIN_NODES = <num>
MAX_NODES = <num>
INSTANCE_FAMILY = <instance_family_name>
[ AUTO_RESUME = { TRUE | FALSE } ]
[ INITIALLY_SUSPENDED = { TRUE | FALSE } ]
[ AUTO_SUSPEND_SECS = <num> ]Pick the instance family to match the workload. A PyTorch job that needs GPUs gets nothing to accelerate on from a pool built on a CPU family. It needs a pool whose instance family is a GPU family. If you don't name a custom pool for ML Jobs, the default uses the CPU_X64_S instance family, with a minimum of 1 node and a maximum of 25. That's enough for general CPU work, not for deep learning.
MAX_NODES also caps how wide one distributed job can go. A multi-node job that asks for a number of target instances needs a pool whose MAX_NODES is at least that number.
A pool's idle cost depends on when it suspends. When no services or jobs are active on any node, Snowflake's auto-suspend suspends the pool to cut costs. AUTO_SUSPEND_SECS on the pool sets how long that idle period lasts first. Suppose a pool has MIN_NODES = 2 and a one-hour window, and serves two job runs a day. It keeps two nodes billed for an hour after each run. Shortening the window cuts that cost directly. AUTO_RESUME lets the next submission wake the pool again.
Checkpoint 1 of 8· Check yourself
A team submits a multi-node ML Job with target_instances=4 to a compute pool created with MIN_NODES = 1 and MAX_NODES = 2. What do they need to change so the pool can support this job?
MAX_NODES caps how far the pool can scale, so it also caps how wide one distributed job can go. A job that asks for 4 target instances needs a pool whose MAX_NODES is at least 4. A bigger instance family changes the hardware on each node, not how many nodes the pool can provide.
“You must set MAX_NODES to be greater than or equal to the number of target instances that you’re using to run your training job.”Source: docs.snowflake.com
2.Running your own container image on a pool
Snowpark Container Services runs a container image that you build. Use it when your code needs something no Snowflake-provided environment has, such as a proprietary C++-backed scoring library or a native system package. The library goes into the image, and the image runs on a compute pool. This is still Snowflake-native: the image is stored, scheduled and billed inside your account. You build the image with ordinary Docker tooling on your machine or in CI. Snowflake doesn't build it for you.
To get an image running on a pool:
- Create a repository with
CREATE IMAGE REPOSITORY. Snowflake's registry is OCIv2-compliant, so the Docker CLI can push to it. - Build the image outside Snowflake, tag it with the repository's fully qualified path and push it with
docker push. - Put that repository path in the
imagefield of a container in the service specification. The pool pulls the image from there.
spec:
container:
- name: main
image: /tutorial_db/data_schema/tutorial_repository/my_job_image:latestOnce the image is in a repository, it runs in one of two forms, and they handle failure in opposite ways. A service runs until you stop it, like a web service. If its container exits for any reason, Snowflake restarts it. A job service has a finite lifespan, like a stored procedure: it runs its work and then ends.
| Property | Service | Job service |
|---|---|---|
| Lifespan | Long-running; you stop it explicitly | Finite, like a stored procedure |
| Container exits | Snowflake restarts the container | The run ends |
| Typical fit | Web UI or API serving requests | Batch training or scoring run |
That difference decides batch training. Take a four-hour nightly run where a partial run corrupts the output table. As a service, it would be restarted after a crash and write a second partial result. As a job service, a crash ends the run and nobody retries it blindly. Put that job service on a pool with a short AUTO_SUSPEND_SECS, and the pool also stops billing soon after the run finishes.
Checkpoint 2 of 8· Put it in order
A scoring service needs a proprietary C++ library that no Snowflake-provided environment includes. The image repository already exists. Put the steps for getting this library running on a compute pool in order.
- 1.Tag the image with the Snowflake image repository's fully qualified path
- 2.Put the repository path in the image field of a container in the service specification
- 3.Push the image to the repository with docker push
- 4.Build the image with ordinary Docker tooling on your machine or in CI
Snowflake doesn't build the image for you. You build it outside Snowflake, tag it with the repository's fully qualified path, then push it. Only after the push can a service specification point at that path, and the pool pulls the image from there.
“Tag the locally built, validated image with the fully qualified URL of the Snowflake image repository, then push it.”Source: docs.snowflake.com
Checkpoint 3 of 8· Exam question
A team needs a proprietary C++-backed scoring library, which is not in the Snowflake conda channel, available inside a training workload on Snowflake. Which approach supports it with the least departure from Snowflake-native operation?
Correct answer: C — Build a Docker image containing the library, push it to a Snowflake image repository, and run it as a service on a compute pool
- A. Incorrect: the Anaconda channel in Snowflake is a curated set of packages; a proprietary library is not in it, and a notebook cell cannot add it to that channel.
- B. Incorrect: stage imports into Snowpark stored procedures only work for pure-Python code, and a compiled native library cannot be loaded this way on a warehouse.
- C. Correct: a container image bundles arbitrary native dependencies, is pushed to a Snowflake image repository, and runs under Snowpark Container Services on a compute pool, so the workload stays inside Snowflake.
- D. Incorrect: Snowpark-optimized warehouses provide more memory per node, not the ability to install arbitrary packages, and warehouse sandboxes do not permit runtime pip installs of native code.
Checkpoint 4 of 8· Exam question
Which TWO steps are required to make a custom container image usable by a service in Snowpark Container Services?(Select 2)
Correct answers: D, E — Reference the pushed image by its repository path in the service specification's container image field, so the pool pulls it; Build the image outside Snowflake, create a repository with CREATE IMAGE REPOSITORY, and push the image to it with Docker
- A. Incorrect: Snowflake does not build images from a Dockerfile in a stage; CREATE SERVICE only references an already built image in the registry.
- B. Incorrect: warehouses do not run user containers and have no image setting; containers run only on compute pools.
- C. Incorrect: external volumes point at cloud object storage for data, not container registries, and they play no role in image distribution.
- D. Correct: the service specification's image field must point at the pushed image path in the repository so that the compute pool pulls the right image.
- E. Correct: Snowflake offers an OCI-compliant registry; images are built outside Snowflake and pushed into an image repository created with CREATE IMAGE REPOSITORY.
Checkpoint 5 of 8· Exam question
A nightly model training run takes four hours and must not be restarted automatically if the process exits with an error, because partial runs corrupt an output table. Which TWO choices fit?(Select 2)
Correct answers: C, D — Run it on a compute pool that has a short AUTO_SUSPEND_SECS so that the pool suspends once the job finishes; A job service started with EXECUTE JOB SERVICE, because exited containers are not restarted and the job has a finite lifetime
- A. Incorrect: regular services are intended for long-running applications and Snowflake restarts containers that exit, which would re-run the partial training.
- B. Incorrect: a service is not dropped automatically when its container exits; it stays and keeps being restarted until explicitly dropped or suspended.
- C. Correct: a short auto-suspend lets the pool release nodes after the finite job ends, so the nightly run is not billed for idle time.
- D. Correct: job services run to completion like stored procedures, and Snowflake does not restart exited job containers, which prevents repeated partial writes.
- E. Incorrect: service functions expose a service endpoint to SQL for inference-style calls and are not a batch training construct with run-once semantics.
3.Snowpark-optimized warehouse or Container Runtime
Snowflake has two places to train a model, and they are different kinds of machine. A Snowpark-optimized warehouse is a virtual warehouse built for jobs that need a lot of memory and compute. Snowpark stored procedures run on it, so it suits single-node training: the procedure loads the data into its own memory, trains, and saves the model to a stage. Container Runtime is a set of versioned, preconfigured ML environments that run on Snowpark Container Services, using CPU or GPU compute pools. They come with popular ML and deep learning frameworks already installed.
The deciding question is whether the workload has to grow beyond one machine. A stored procedure on a warehouse trains on one node. When you use the Snowflake ML APIs in Container Runtime, it spreads the processing across the available resources. By default it uses every GPU on a multi-GPU node, and it can run across several nodes. Container Runtime also lets you pin a runtime version, so a workload keeps the same environment until you choose to upgrade.
| Consideration | Snowpark-optimized warehouse | Container Runtime on a compute pool |
|---|---|---|
| Execution model | Snowpark stored procedure on a single node | Notebook or ML Job on Snowpark Container Services, one or many nodes |
| Hardware | High memory and compute, no GPU choice | CPU or GPU instance families |
| Scaling training | Limited to one stored procedure's memory | Distributed across nodes and GPUs |
| Wins when | Single-node training that already runs as SQL-orchestrated procedures | GPU work, data larger than one node's memory, multi-node jobs |
Use each kind of compute for what it does best. Container workloads still send their SQL to a warehouse. For example, a service specification can pass a SNOWFLAKE_WAREHOUSE to its container. So moving training onto a pool doesn't remove the warehouse. Size the warehouse for the queries and the pool for the model fitting.
Checkpoint 6 of 8· Check yourself
Which of these is a good reason to move model training off a Snowpark-optimized warehouse and onto Container Runtime?
A Snowpark-optimized warehouse gives a stored procedure lots of memory on one node, but you can't choose GPUs. Container Runtime runs on GPU compute pools, and by default the Snowflake ML APIs use every GPU on a multi-GPU node. The job's SQL still runs on a warehouse, so moving to a pool doesn't remove the warehouse.
“By default, this framework uses all GPUs on multi-GPU nodes”Source: docs.snowflake.com
4.Distributed training APIs in each runtime
Container Runtime comes with distributed trainers in the snowflake.ml.modeling.distributors namespace. They are distributed versions of XGBoost, LightGBM and PyTorch, and their APIs look like the standard libraries. A Snowflake-provided Ray cluster runs them, so you don't configure a cluster. They find and use all the cluster's resources on their own. A warehouse stored procedure has no equivalent: its training stays inside the memory of one procedure.
| Framework | Primary class | Scaling configuration |
|---|---|---|
| XGBoost | XGBEstimator | XGBScalingConfig |
| LightGBM | LightGBMEstimator | LightGBMScalingConfig |
| PyTorch | PyTorchDistributor | PyTorchScalingConfig |
The distributed XGBEstimator differs from open-source XGBoost in three ways. The training parameters go to the constructor through params. A data connector is passed straight to fit along with input_cols and label_col (the connector itself is covered with DataConnector ingestion). The scaling config is optional, because Snowflake uses all available resources by default. Set it only to override something, for example use_gpu=True.
from snowflake.ml.modeling.distributors.xgboost import XGBEstimator, XGBScalingConfig
# Training parameters
params = {
'objective': 'reg:squarederror',
'max_depth': 6,
'learning_rate': 0.1
}
# Automatic scaling (recommended)
estimator = XGBEstimator(
params=params
)
# Call with custom GPU scaling
gpu_estimator = XGBEstimator(
params=params,
scaling_config=XGBScalingConfig(use_gpu=True) # optional - available resources will be used automatically
)Not every Container Runtime workload needs a distributor. Plain open-source libraries are the better choice for small datasets on a single node, for quick prototypes, and for lift-and-shift code that doesn't need to scale. Switch to the distributed trainers when the data is larger than one node's memory or the job should use several GPUs.
Checkpoint 7 of 8· Fill the gap
This distributed XGBoost sample trains on all available resources by default. The second estimator overrides that default to train on GPUs. Which scaling-config argument goes in the blank?
from snowflake.ml.modeling.distributors.xgboost import XGBEstimator, XGBScalingConfig
# Training parameters
params = {
'objective': 'reg:squarederror',
'max_depth': 6,
'learning_rate': 0.1
}
# Automatic scaling (recommended)
estimator = XGBEstimator(
params=params
)
# Call with custom GPU scaling
gpu_estimator = XGBEstimator(
params=params,
scaling_config=XGBScalingConfig( ? =True) # optional - available resources will be used automatically
)XGBScalingConfig is optional because Snowflake uses all available resources by default. You set it only to override that default, and use_gpu=True is the override that makes training use GPUs. The number of nodes and the compute pool are set when you submit the job, not in the estimator's scaling config.
Source: docs.snowflake.com5.Getting a library Snowpark doesn't provide into a runtime
Whether Snowpark provides a library decides which runtime a training job can use. A Python stored procedure on a warehouse gets its libraries from the PACKAGES clause in its definition, for example PACKAGES = ('snowflake-snowpark-python', 'scikit-learn', 'joblib'). Some libraries can't be named there: a package from your organisation's internal repository, a version pinned for an audit, or a library that needs a native system package. Those have to reach the job inside a container image that runs on a compute pool.
There are three ways to put the library into a container, and each one wins in a different setting:
- Install at run time. Container Runtime comes with popular ML and deep learning frameworks, and you can add packages from public PyPI or from an internally hosted package repository. This is the quickest way to try a library. The cost is that every Notebook session repeats the
pip install, and the container needs an external access integration, because containers have no internet access by default. - Custom runtime environment. You extend a Snowflake base image with the library and register it. A notebook or ML Job then selects it with
runtime_environment="cre@<name>". Building and registering the image is a separate procedure. This route wins in a regulated account that only accepts scanned, approved images, or in one with no route to PyPI at all. - Your own image on Snowpark Container Services. A service or job service runs whatever image you push to a repository. Use it when the library can't live alongside Container Runtime's own stack. A custom runtime image must be derived from a Snowflake base image, and its Python environment must resolve with no package conflicts.
Pinning the environment matters as much as pinning the packages. An ML Job submitted without runtime_environment runs on the latest Container Runtime version available on its pool. A job that passed last quarter can therefore start on a newer image with different library versions. Passing a version string such as "2.3.0", or a cre@ name, holds the job on one environment. Services work the other way round. CREATE SERVICE records the exact image version, and Snowflake keeps deploying that version when the service resumes, scales out or restarts for maintenance, even after a newer image is pushed. Only ALTER SERVICE … <fromSpecification> picks up the newer image.
Build every image for linux/amd64. Snowpark Container Services runs only linux/amd64 images, so pass --platform linux/amd64 to docker build, even when the build machine has an ARM chip.
Checkpoint 8 of 8· Check yourself
A bank's fraud team needs a gradient-boosting package that their stored procedure's PACKAGES list doesn't offer. Security policy allows only scanned, approved images, and the account has no external access integration to PyPI. How should the package reach their ML Jobs?
Without an external access integration to PyPI, a run-time pip install has nowhere to download from. An approved custom image already contains the package, so nothing is fetched at run time, and the bank's image-scanning policy is met.
“Organizations that don’t have external access integrations to PyPI repositories can use org-approved and managed images without needing PyPI access at runtime.”Source: docs.snowflake.com
Sources
(11)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“Specifies the minimum number of nodes for the compute pool. This value must be greater than 0.”
↩︎ Compute pools: the hardware behind Container Runtime“the number of credits consumed while the compute pool is running”
↩︎ Compute pools: the hardware behind Container Runtime“Identifies the type of machine you want to provision for the nodes in the compute pool.”
↩︎ Key concept“Specifies the minimum number of nodes for the compute pool. This value must be greater than 0.”
↩︎ Exam trap 1 - 2.
“The default compute pool size uses the CPU_X64_S instance family. The minimum number of nodes is 1 and the maximum is 25.”
↩︎ Compute pools: the hardware behind Container Runtime“To use a custom image, reference it with the runtime_environment="cre@<name>" format.”
↩︎ Getting a library Snowpark doesn't provide into a runtime“Snowflake automatically uses the latest available version of the Snowflake Container Runtime on your compute pool.”
↩︎ Getting a library Snowpark doesn't provide into a runtime“Snowflake automatically uses the latest available version of the Snowflake Container Runtime on your compute pool.”
↩︎ Exam trap 3 - 3.https://docs.snowflake.com/en/developer-guide/snowpark-container-services/working-with-servicesOfficial docs
“When no services or jobs are active on any compute pool node, Snowflake’s compute pool auto-suspend mechanism suspends the pool to reduce costs.”
↩︎ Compute pools: the hardware behind Container Runtime“Snowflake records the specific version of the provided image.”
↩︎ Getting a library Snowpark doesn't provide into a runtime - 4.
“You must set MAX_NODES to be greater than or equal to the number of target instances that you’re using to run your training job.”
↩︎ Compute pools: the hardware behind Container Runtime - 5.
“You create a repository using the CREATE IMAGE REPOSITORY command.”
↩︎ Running your own container image on a pool“After you upload your application image to a repository, you can run your application by creating a long-running service or executing a job service.”
↩︎ Running your own container image on a pool“A job service has a finite lifespan”
↩︎ Running your own container image on a pool“If a service container exits for whatever reason, Snowflake restarts that container.”
↩︎ Exam trap 2 - 6.
“Build the custom image locally using standard Docker tooling.”
↩︎ Running your own container image on a pool“Custom images eliminate the need for manual pip install commands in every Notebook session”
↩︎ Getting a library Snowpark doesn't provide into a runtime“All custom images must be derived from an official Snowflake ML runtime base image.”
↩︎ Getting a library Snowpark doesn't provide into a runtime“The Python environment must have a fully resolved dependency tree with no package conflicts.”
↩︎ Getting a library Snowpark doesn't provide into a runtime“Tag the locally built, validated image with the fully qualified URL of the Snowflake image repository, then push it.”
↩︎ Checkpoint“Organizations that don’t have external access integrations to PyPI repositories can use org-approved and managed images without needing PyPI access at runtime.”
↩︎ Checkpoint - 7.https://docs.snowflake.com/en/developer-guide/snowpark/python/python-snowpark-training-mlOfficial docs
“Snowpark-optimized warehouses make it possible to use Snowpark stored procedures to run single-node ML training workloads directly in Snowflake.”
↩︎ Snowpark-optimized warehouse or Container Runtime“PACKAGES = ('snowflake-snowpark-python', 'scikit-learn', 'joblib')”
↩︎ Getting a library Snowpark doesn't provide into a runtime - 8.
“which offers the ability to run on CPU or GPU compute pools”
↩︎ Snowpark-optimized warehouse or Container Runtime“By default, this framework uses all GPUs on multi-GPU nodes”
↩︎ Snowpark-optimized warehouse or Container Runtime“These are found in the snowflake.ml.modeling.distributors namespace.”
↩︎ Distributed training APIs in each runtime“you can import packages from external sources like public PyPI repositories, or internally-hosted package repositories”
↩︎ Getting a library Snowpark doesn't provide into a runtime - 9.
“The Container Runtime environment is backed by a Ray cluster provided by Snowflake, with no user configuration required”
↩︎ Distributed training APIs in each runtime“Train models on datasets that are larger than the memory of a single compute node”
↩︎ Distributed training APIs in each runtime - 10.
“By default, application containers don’t have permission to access the internet.”
↩︎ Getting a library Snowpark doesn't provide into a runtime - 11.https://docs.snowflake.com/en/developer-guide/snowpark-container-services/specification-referenceOfficial docs
“Currently, Snowpark Container Services requires linux/amd64 platform images.”
↩︎ Getting a library Snowpark doesn't provide into a runtime