What you will be able to do
- Explain where Workspace notebook code runs and which compute pools are available without setup
- State how much compute, including GPUs, a single notebook service gets, and why concurrent users need more than one node
- Predict what survives idle-timeout suspension and scheduled maintenance of a notebook service
- Install open-source packages in Container Runtime and recognise when pip stops being enough
- Load Snowflake data with DataConnector into pandas, PyTorch, TensorFlow or a distributed trainer
- Decide when to develop in a Workspace notebook and when to use a Python file, SQL or a stored procedure
- Identify the default GPU allocation constraints for an individual notebook instance, including the single GPU a node of the default AWS GPU pool provides
- Use DataConnector to speed up data ingestion into open source packages such as pandas, scikit-learn, XGBoost and PyTorch
Key concept
Notebook service — The Snowflake-managed container that hosts the kernel for a Workspace notebook or Python file. It belongs to one user and occupies one node of a CPU or GPU compute pool. All notebooks attached to it share that node.
1.Where Workspace notebook code runs
Snowflake Notebooks in Workspaces don't run on a virtual warehouse or in the browser. They run on Snowflake Container Runtime, which is a set of preconfigured ML environments on Snowpark Container Services. That compute comes from compute pools of CPU or GPU nodes. This matters for planning because GPU access comes from picking a GPU compute pool. Resizing a warehouse never gives you a GPU, and any Python package can be layered on top of the runtime image, not only those from a curated channel.
When you run a notebook or a .py file in a workspace, you create a notebook service. This is a Snowflake-managed container that hosts the kernel and runs your code. When you create it, you choose the Python version (3.10 to 3.12), the Container Runtime version, the compute pool, the idle timeout and any external access integrations. A warehouse still matters, but only for the SQL and Snowpark queries that the notebook pushes down. The Python itself runs in the container.
Every account gets two system compute pools, so notebooks work without any setup: SYSTEM_COMPUTE_POOL_CPU and SYSTEM_COMPUTE_POOL_GPU. They start suspended and only cost money once a notebook, ML job, model-serving or Streamlit workload starts using them. Nobody has to run CREATE COMPUTE POOL before the first notebook. Some regions don't offer the GPU pool. Examples are Singapore, Paris and Osaka on AWS, Central US on Azure, and all of Google Cloud.
| Pool | Instance family | MIN_NODES / MAX_NODES | AUTO_SUSPEND_SECS |
|---|---|---|---|
| SYSTEM_COMPUTE_POOL_CPU | CPU_X64_S | 1 / 150 | 259200 |
| SYSTEM_COMPUTE_POOL_GPU | GPU_NV_S (AWS), GPU_NV_SM (Azure) | 1 / 50 | 600 |
Start on a small CPU pool and move to a higher-memory GPU pool for GPU-heavy work such as computer vision or large language and vision models. Administrators who want to see spend per role can give each role its own compute pool and put a budget on it.
Data ingestion runs on the same Container Runtime nodes, and it is often where ML work first stalls. A notebook that reads a ten-million-row table straight into pandas pushes every row through one process before scikit-learn or XGBoost sees any of it. DataConnector speeds up that ingestion by parallelizing the reads across the runtime's compute nodes, then hands the data to open source packages as a pandas DataFrame, a PyTorch dataset or a TensorFlow dataset. Only the load step changes, to something like DataConnector.from_dataframe(session.table(...)).to_pandas(). The open-source training and evaluation code after it stays exactly as written.
Checkpoint 1 of 10· Check yourself
A data scientist in an AWS US East account opens a new notebook in Workspaces to fine-tune a PyTorch vision model and needs GPU acceleration. No one has set up any compute for notebooks yet. What should they do?
Notebook Python runs on Container Runtime in Snowpark Container Services, so GPUs come from choosing a GPU compute pool, not from any warehouse setting. Every account already has SYSTEM_COMPUTE_POOL_GPU (in regions that offer it), so nobody has to create a pool first.
“occur within Snowpark Container Services, which offers the ability to run on CPU or GPU compute pools.”Source: docs.snowflake.com
Checkpoint 2 of 10· Exam question
A data science lead asks what compute foundation Snowflake Notebooks in Workspaces use so the team can plan GPU access and package flexibility. Which statement is accurate?
Correct answer: C — Notebooks run on Container Runtime, which is powered by Snowpark Container Services and runs on compute pools of CPU or GPU nodes.
- A. Incorrect. Python code runs on a container service in Snowflake, not in the browser, so heavy training does not depend on the client machine.
- B. Incorrect. Virtual warehouses have no GPU option; GPU workloads need a GPU compute pool, and resizing a warehouse only adds CPU and memory.
- C. Correct. Container Runtime is built on Snowpark Container Services, so notebooks execute on compute pool nodes rather than on a virtual warehouse.
- D. Incorrect. The stored procedure sandbox is the warehouse model; Container Runtime notebooks can also install packages from PyPI via external access integrations.
Checkpoint 3 of 10· Exam question
An administrator wants to know which compute pools exist in every Snowflake account for notebook workloads without any setup. Which pair is correct?
Correct answer: D — SYSTEM_COMPUTE_POOL_CPU and SYSTEM_COMPUTE_POOL_GPU, provisioned automatically for notebooks on Container Runtime.
- A. Incorrect. These names do not exist; notebooks rely on the two SYSTEM_COMPUTE_POOL_* pools.
- B. Incorrect. Notebook Container Runtime uses compute pools, not warehouses, and no such warehouses are created.
- C. Incorrect. The system pools already exist; creating a custom pool is optional and uses names you choose.
- D. Correct. Snowflake provisions one CPU and one GPU system compute pool per account for notebook use.
2.One node per notebook service: the default GPU allocation
In the Workspaces environment, the default GPU allocation for an individual notebook instance is exactly one node of the selected compute pool. The notebook service gets the GPUs on that node and no more, because a service never spans nodes. The instance family of the pool therefore sets the ceiling. Every notebook or Python file attached to the same service shares that node, so a second notebook doesn't bring extra GPUs; it competes for the same ones. To give a workload dedicated GPUs, create a separate notebook service for it and attach nothing else.
On the default GPU pool in AWS, that ceiling is a single GPU. SYSTEM_COMPUTE_POOL_GPU uses the GPU_NV_S instance family there, and a GPU_NV_S node has 27 GB of memory and one GPU. No change to the notebook code lets an individual notebook instance on that pool address a second GPU. A workload that needs several GPUs on one machine needs a pool built on a multi-GPU instance family, or a multi-node cluster.
On that one node, Snowflake's distributed ML APIs use everything available. By default, the distributed framework under the Snowflake ML modeling and data-loading APIs uses all GPUs on a multi-GPU node. Plain open-source code, such as a PyTorch loop that targets a single CUDA device, only uses the GPUs it addresses. The PyTorch distributor is the exception to the use-everything default: its scaling configuration reserves no GPUs per worker until you set num_gpus. To go beyond one node, use Container Runtime 2.3 or later, which supports multi-node clusters for PyTorch, XGBoost and LightGBM. A notebook service doesn't do this by itself.
The one-node rule also limits how many people can use a pool at once. Each service belongs to one user and takes one node, and each compute node runs only one notebook per user at a time. So a custom pool with MAX_NODES = 1 can host only one person's notebook at a time, and the next user has nothing to start on. Granting more privileges or changing a warehouse setting won't fix this. Set MAX_NODES to at least the number of users who need to work at the same time. The system GPU pool allows up to 50 nodes for this reason.
Checkpoint 4 of 10· Check yourself
Three analysts share a custom GPU compute pool created with MAX_NODES = 1. The first analyst's notebook service is running. The other two can't start theirs, even though all three roles have USAGE on the pool. What fixes this?
Each notebook service belongs to a single user and takes up one whole node. With MAX_NODES = 1, only one user's service can run at a time. More privileges or a bigger warehouse don't add nodes, and a user can't join another user's service.
“Each notebook service is scoped to a single user and occupies one node on the selected compute pool.”Source: docs.snowflake.com
Checkpoint 5 of 10· Exam question
Three data scientists share a custom compute pool with MAX_NODES = 1 for their Workspaces notebooks. The second and third users cannot start their notebooks while the first is running. What is the cause and the fix?
Correct answer: B — One notebook runs per user on each node, so MAX_NODES must exceed one to give concurrent users separate nodes.
- A. Incorrect. There is no account-wide limit of one notebook; the constraint is node capacity within the pool.
- B. Correct. Notebook services are limited per node, and the pool needs additional nodes to host other users' notebooks at once.
- C. Incorrect. Missing privileges would block access to objects, not prevent a pool from scheduling a second user's service.
- D. Incorrect. Notebooks on Container Runtime do not use warehouse clusters; capacity comes from compute pool nodes.
3.Suspension, idle timeout and maintenance
By default, a notebook service suspends after 24 hours of idle time. You can change this when you create or edit the service, and changing it doesn't restart the service. Idle time starts only when no cells are running in any attached notebook. If several notebooks share the service, the clock starts when the last one goes idle. Administrators can limit the timeout values users can pick with the NOTEBOOK_VNEXT_IDLE_TIMEOUT_OPTIONS_MINUTES account parameter.
Suspension is a hard reset of the container. All attached notebooks disconnect, and in-memory state, installed packages and variables are lost. Files that code or the terminal wrote to the Workspace file system or to /tmp are lost as well. To resume, connect a notebook to the service or run one that was connected before. Then rerun the cells that load data and install packages. You can also suspend the service by hand from the Connected menu.
Notebook services are Snowpark Container Services, so they also need periodic maintenance. Once a service reaches RUNNING, whether newly created or resumed, it won't be disrupted by maintenance for seven calendar days (168 hours). After that, Snowflake may suspend it for mandatory maintenance, which typically takes about five minutes and restarts the service. A training session started on a Friday will not still be holding state ten days later. Long runs should save checkpoints to durable storage, or run as a job instead.
A service also stays on the Runtime version it was created with. A service on Runtime 2.0 doesn't upgrade when newer versions come out. You upgrade by editing the service, and editing the runtime or the external access integrations restarts it. New services can't select a Runtime with known high or critical CVEs, but existing services can keep using one.
Checkpoint 6 of 10· Check yourself
On a Monday, an ML engineer starts a new notebook service and launches a hyperparameter search that keeps cells running for two weeks straight. All intermediate results stay in memory. What should they expect?
Running cells only stop the idle-timeout clock. Maintenance follows its own schedule: the service is protected for 168 hours after it reaches RUNNING, and after that it may be suspended and restarted. That restart clears memory, packages and /tmp. Neither the idle-timeout setting nor Runtime pinning stops it.
“Seven days after creation, the service may be suspended for mandatory maintenance.”Source: docs.snowflake.com
4.Open-source packages in Container Runtime
The default Container Runtime comes with about 100 preinstalled data-science and ML packages, plus first-party Snowflake packages such as snowflake-snowpark-python. scikit-learn, numpy, scipy, XGBoost and LightGBM are ready to import. The CPU and GPU runtimes don't contain the same packages. The GPU image is the one for deep learning with PyTorch or TensorFlow. To check what a runtime contains, run !pip freeze in a cell.
Missing packages are added with !pip install, either in a cell or in the notebook terminal. Where the package comes from determines what setup you need:
| Option | When it fits | What the user does |
|---|---|---|
| Uploaded .whl or .py files (Workspace or stage) | Private or curated packages; use a Shared workspace or stage for team use | !pip install file_name.whl, or import the .py module |
| Artifact repositories | Extra PyPI packages with package policy enforcement | !pip install; the Snowflake PyPI repo is available to PUBLIC by default |
| External Access Integrations (EAIs) | Reaching external repos such as PyPI, Artifactory or GitHub | Admin sets up the EAI, then the user runs !pip install |
| Custom images | Full control: system-level dependencies, security tooling, standardised images | Nothing, once the right image is selected |
pip adds packages to the base image. It doesn't let you swap the base image's versions. If a package already ships in the base image, installing a different version with pip doesn't change it. A requirements.txt that pins versions in conflict with the preinstalled packages can break the Python environment, so check compatibility before running !pip install -r requirements.txt. A wheel whose dependencies aren't installed needs its whole dependency tree uploaded, or an EAI to a repository that has them.
pip stops being enough in three situations. The first is when you need a version that conflicts with the base image. The second is when you need system-level dependencies that a wheel can't supply. The third is when the reinstall after every suspension becomes a burden, because suspension removes everything you pip-installed. In those cases the platform team builds a custom container runtime image, and it becomes the base environment. Startup may be slower, and images built on old runtimes can lose features and performance.
Checkpoint 7 of 10· Match them up
A platform team is choosing how users will add packages on top of the default Container Runtime. Match each option to the situation it fits best.
Tap a term, then the definition that fits it.
Uploaded files suit private or curated packages. Artifact repositories add PyPI packages with policy enforcement. EAIs give pip a route to external repositories. A pip-based option can't supply system-level dependencies or change the versions in the base image, so those cases need a custom image, which becomes the base environment.
“Use when you require full customization and control over the container environment, including preinstalled system-level dependencies, custom security tooling, or standardized images”Source: docs.snowflake.com
5.DataConnector: fast loading into open-source objects
Reading a large table straight into a pandas DataFrame is slow, because one process pulls all the rows. DataConnector (snowflake.ml.data) parallelises the reads across the compute nodes. It then hands the data to the framework you are using as a pandas DataFrame, a PyTorch dataset or a TensorFlow dataset. It works in Workspace notebooks and in ML Jobs, because both run on Container Runtime, which uses Ray to spread the work across nodes.
| Method | Purpose |
|---|---|
| DataConnector.from_dataframe | Build from a Snowpark DataFrame (direct table access; best during development) |
| DataConnector.from_dataset | Build from a Snowflake ML Dataset by name and version (versioned; best for production) |
| DataConnector.from_sources | Build from a list of DataFrames and/or Datasets |
| data_connector.to_torch_dataset | Return a PyTorch Dataset |
| data_connector.to_tf_dataset | Return a TensorFlow Dataset, loaded in a streaming fashion |
from snowflake.ml.data.data_connector import DataConnector
from snowflake.snowpark.context import get_active_session
session = get_active_session()
# Create DataConnector from a Snowflake table
data_connector = DataConnector.from_dataframe(session.table("example-table-name"))to_pandas() feeds scikit-learn and other pandas-based libraries. to_torch_dataset(batch_size=32) goes into a PyTorch DataLoader with batch_size=None, because the connector already batches. to_tf_dataset takes batch_size, shuffle and drop_last_batch. DataConnector handles structured data from tables and Datasets. For CSV, Parquet or images in a stage, use the DataSource APIs instead.
Converting to pandas is not the fastest path. For the best performance, pass the DataConnector itself to Snowflake's distributed training APIs, so the data never has to fit in one pandas object. For distributed PyTorch, ShardedDataConnector gives each worker process its own shard of the data.
Checkpoint 8 of 10· Fill the gap
This notebook cell builds a DataConnector over a table that is read through session.table(), which returns a Snowpark DataFrame, during development. Which class method completes it?
from snowflake.ml.data.data_connector import DataConnector
from snowflake.snowpark.context import get_active_session
session = get_active_session()
# Create DataConnector from a Snowflake table
data_connector = DataConnector. ? (session.table("example-table-name"))session.table() returns a Snowpark DataFrame, so from_dataframe is the right method. It gives direct access to the table and is the best choice during development. from_dataset takes a versioned Snowflake ML Dataset by name and version, and from_sources takes a list of sources. to_pandas is an output method that you call on a connector after you build it, not a way to build one.
Source: docs.snowflake.com6.Training and evaluating models with open-source and distributed APIs
Training in a Workspace notebook starts as ordinary open-source code. You load with DataConnector, split with scikit-learn's train_test_split, fit an xgb.XGBClassifier or a LogisticRegression, and score on the held-out set. All of it runs in the container, next to governed data that never leaves Snowflake. This works as long as the data fits in the memory of the one node the service has.
When the data or the model outgrows one node, switch to the distributed classes in snowflake.ml.modeling.distributors: XGBEstimator, LightGBMEstimator and the PyTorch distributor. They are designed to look like the standard APIs, plus a scaling config. With a DataConnector, input_cols and label_col have to be passed explicitly.
# Create Snowflake XGBoost estimator
snowflake_est = XGBEstimator(
n_estimators=1,
objective="reg:squarederror",
scaling_config=XGBScalingConfig(use_gpu=False),
)
# Train using the data connector
# When using a data connector, input_cols and label_col must be provided
fit_booster = snowflake_est.fit(
data_connector,
input_cols=NUMERICAL_COLS,
label_col=LABEL_COL
)For deep learning, the usual fix for slow training is the hardware, not the code. A transformer fine-tune that crawls on a CPU runtime should move to a GPU runtime on a GPU compute pool, where PyTorch and TensorFlow can use CUDA. Use torch.device("cuda" if torch.cuda.is_available() else "cpu") so the same notebook runs on either. Hyperparameter search can use Snowflake's distributed HPO or open-source libraries such as hyperopt and optuna. Ray is also available for scaling arbitrary Python across nodes.
Checkpoint 9 of 10· Fill the gap
To get the best performance, an engineer trains Snowflake's distributed XGBoost on a table that is already loaded into a DataConnector named data_connector. What should be passed as the first argument to fit()?
# Create Snowflake XGBoost estimator
snowflake_est = XGBEstimator(
n_estimators=1,
objective="reg:squarederror",
scaling_config=XGBScalingConfig(use_gpu=False),
)
# Train using the data connector
# When using a data connector, input_cols and label_col must be provided
fit_booster = snowflake_est.fit(
? ,
input_cols=NUMERICAL_COLS,
label_col=LABEL_COL
)For the best performance, pass the DataConnector itself to Snowflake's distributed training APIs. Then the data never has to fit into a single pandas object. Because the estimator gets a DataConnector, input_cols and label_col have to be provided explicitly. Converting to pandas or a PyTorch dataset first is the slower path.
Source: docs.snowflake.com7.When to develop in a Workspace notebook, a Python file, SQL or a stored procedure
Use a Workspace notebook when the work is interactive and iterative: exploring data, testing features, comparing models and looking at results cell by cell. A notebook can cover the whole ML lifecycle, from reading source data to inference, under the governance already on the data. It also gives you the two things other surfaces don't: open-source packages on Container Runtime, and CPU or GPU nodes from a compute pool.
Use a Python file in the same workspace once the logic is stable and should run from start to finish. Running a .py file executes the whole file in one pass, as an IDE or terminal would, on the same notebook services. This suits utilities, training scripts and ETL jobs you also deploy to production. Shared helpers can live in workspace .py modules and be imported into notebooks with from my_utils import my_func.
Use SQL files or stored procedures for logic that is set-based work against tables and needs no notebook kernel, GPU or open-source training library. Snowpark and SQL queries run on a warehouse anyway, so wrapping them in a notebook service only keeps a pool node busy. To move a notebook into production, use the native scheduler, call it from an orchestration script, or run the workload as an ML Job, which runs ML work asynchronously from any development environment.
Checkpoint 10 of 10· Match them up
Your team is dividing an ML project's work among the development surfaces in Snowflake. Match each surface to the piece of work it suits best.
Tap a term, then the definition that fits it.
Notebooks are for interactive, iterative work and add open-source packages plus compute-pool CPUs and GPUs. A .py file runs stable logic end to end on notebook services. Pure set-based table logic runs on a warehouse anyway, so it belongs in SQL or a stored procedure. ML Jobs run ML work asynchronously from any environment.
“Snowflake Notebooks: Interactive development environment for exploring data and building ML models.”Source: docs.snowflake.com
Sources
(11)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“occur within Snowpark Container Services, which offers the ability to run on CPU or GPU compute pools”
↩︎ Where Workspace notebook code runs“Once loaded, the data can be processed using open source packages, or any of the Snowflake ML APIs”
↩︎ Where Workspace notebook code runs“By default, this framework uses all GPUs on multi-GPU nodes”
↩︎ One node per notebook service: the default GPU allocation“The CPU container runtime has different packages than the GPU container runtime.”
↩︎ Open-source packages in Container Runtime“The APIs of the distributed classes are similar to those of the standard versions.”
↩︎ Training and evaluating models with open-source and distributed APIs“occur within Snowpark Container Services, which offers the ability to run on CPU or GPU compute pools”
↩︎ Exam trap 1“occur within Snowpark Container Services, which offers the ability to run on CPU or GPU compute pools.”
↩︎ Checkpoint - 2.
“Snowflake automatically provisions two compute pools in each account for running notebooks: SYSTEM_COMPUTE_POOL_CPU and SYSTEM_COMPUTE_POOL_GPU.”
↩︎ Where Workspace notebook code runs“Note that each compute node is limited to running one notebook per user at a time.”
↩︎ One node per notebook service: the default GPU allocation“you can’t change the version on the package by installing a different version with pip install”
↩︎ Open-source packages in Container Runtime“you can’t change the version on the package by installing a different version with pip install”
↩︎ Exam trap 5 - 3.https://docs.snowflake.com/en/user-guide/ui-snowsight/notebooks-in-workspaces/notebooks-in-workspaces-compute-setupOfficial docs
“the user creates a Snowflake-managed notebook service to host the notebook kernel and execute code”
↩︎ Where Workspace notebook code runs“If a notebook or Python file requires dedicated compute resources, create a separate notebook service”
↩︎ One node per notebook service: the default GPU allocation“This notebook is optimized for Snowflake Container Runtime 2.3 or above, which introduces support for multi-node clusters.”
↩︎ One node per notebook service: the default GPU allocation“By default, notebook services have an idle timeout of 24 hours.”
↩︎ Suspension, idle timeout and maintenance“If multiple notebooks share the same service, idle time starts only when the last notebook becomes idle (no cells running).”
↩︎ Suspension, idle timeout and maintenance“Maintenance typically takes about five minutes and suspends and restarts the notebook service.”
↩︎ Suspension, idle timeout and maintenance“Each notebook service is scoped to a single user and occupies one node on the selected compute pool.”
↩︎ Key concept“Each notebook service is scoped to a single user and occupies one node on the selected compute pool.”
↩︎ Exam trap 2“it is guaranteed not to be disrupted for seven calendar days (168 hours) due to service maintenance.”
↩︎ Exam trap 4“All notebooks and Python files connected to the same service share the compute resources on that node.”
↩︎ Prediction“Each notebook service is scoped to a single user and occupies one node on the selected compute pool.”
↩︎ Checkpoint“Suspending a service disconnects all notebooks connected to it, clears in-memory states, and removes all packages and variables.”
↩︎ Prediction“Seven days after creation, the service may be suspended for mandatory maintenance.”
↩︎ Checkpoint - 4.https://docs.snowflake.com/en/developer-guide/snowpark-container-services/working-with-compute-poolOfficial docs
“With system compute pools, you can run these workloads immediately, no compute pool setup required.”
↩︎ Where Workspace notebook code runs“In Azure, GPU_NV_SM. In AWS, GPU_NV_S.”
↩︎ One node per notebook service: the default GPU allocation - 5.
“take advantage of Snowflake’s distributed processing to accelerate data ingestion for your training and inference workflows”
↩︎ Where Workspace notebook code runs“The DataConnector accelerates data loading by parallelizing the reads across multiple compute nodes.”
↩︎ DataConnector: fast loading into open-source objects“pass a DataConnector directly to Snowflake’s optimized distributed training APIs instead of converting to pandas, PyTorch, or TensorFlow datasets first”
↩︎ DataConnector: fast loading into open-source objects“Run your ML workloads asynchronously from any development environment.”
↩︎ When to develop in a Workspace notebook, a Python file, SQL or a stored procedure“pass a DataConnector directly to Snowflake’s optimized distributed training APIs instead of converting to pandas, PyTorch, or TensorFlow datasets first”
↩︎ Exam trap 6“Snowflake Notebooks: Interactive development environment for exploring data and building ML models.”
↩︎ Checkpoint - 6.https://docs.snowflake.com/en/developer-guide/snowpark-container-services/specification-referenceOfficial docs
“You create a compute pool of two nodes; each node has 27 GB of memory and one GPU”
↩︎ One node per notebook service: the default GPU allocation“You create a compute pool of two nodes; each node has 27 GB of memory and one GPU”
↩︎ Exam trap 3 - 7.https://docs.snowflake.com/en/developer-guide/snowflake-ml/container-runtime-multi-nodeOfficial docs
“num_gpus: The number of GPUs to reserve for each worker. The default is 0, indicating no GPUs are reserved.”
↩︎ One node per notebook service: the default GPU allocation - 8.
“All notebook services are pinned to the Runtime selected at creation unless you explicitly change it by editing the service.”
↩︎ Suspension, idle timeout and maintenance“preinstalled with ~100 popular data science and machine learning packages and first-party Snowflake packages such as snowflake-snowpark-python”
↩︎ Open-source packages in Container Runtime“If the package version specified in requirements.txt conflicts with supported versions of the pre-installed packages, the Python environment may break.”
↩︎ Open-source packages in Container Runtime“Use when you require full customization and control over the container environment, including preinstalled system-level dependencies, custom security tooling, or standardized images”
↩︎ Checkpoint - 9.
“can be slow, especially when you’re trying to read directly into an object such as a pandas dataframe”
↩︎ DataConnector: fast loading into open-source objects“You can use a GPU-powered container runtime image to train deep learning models with PyTorch, TensorFlow, and other frameworks.”
↩︎ Training and evaluating models with open-source and distributed APIs“Training ML models on large datasets can exceed the resources of a single node.”
↩︎ Training and evaluating models with open-source and distributed APIs - 10.
“for example, utilities, training scripts, or ETL jobs you also deploy to production”
↩︎ When to develop in a Workspace notebook, a Python file, SQL or a stored procedure - 11.https://docs.snowflake.com/en/user-guide/ui-snowsight/notebooks-in-workspaces/notebooks-in-workspaces-overviewOfficial docs
“Use the native scheduler or incorporate notebooks into orchestration scripts for production pipelines.”
↩︎ When to develop in a Workspace notebook, a Python file, SQL or a stored procedure“consolidate their complete ML lifecycle, from source data access to model inference, within a single Jupyter notebook environment”
↩︎ When to develop in a Workspace notebook, a Python file, SQL or a stored procedure