What you will be able to do
- Create an experiment, open runs with start_run, and explain when a run becomes immutable and which privileges this requires
- Choose correctly between log_param, log_metric with a step, log_artifact and log_model when recording training metadata
- Compare, query, retrieve and clean up runs, and recognise the per-schema, per-experiment and per-run limits
- Configure a Tuner with a search algorithm, sampling functions and TunerConfig, and read its TunerResults
- Wrap an external model in a CustomModel subclass using ModelContext and @inference_api methods
- Tie a registered model version to the exact Dataset version it was trained on and trace that lineage
Key concept
Experiment run — A run is one training attempt inside a named experiment. It holds the parameters, step-indexed metrics, artifacts and linked model versions from that attempt, and it is frozen once completed. Comparing, tuning and reproducing models in Snowflake ML all come down to comparing runs.
1.Experiments, runs and the privileges behind them
Snowflake ML Experiments give model development a structure: a schema contains experiments, an experiment contains runs, and each run holds its own parameters, metrics and artifacts. One experiment usually covers one modelling question, such as which gradient-boosting setup best predicts churn. Each attempt at answering it is a run. Snowsight builds its run lists, charts and comparisons from this metadata. Snowflake doesn't restrict what you store as a run artifact, so anything that helps you evaluate a model can go in: a confusion-matrix image, a feature-importance file or a log.
Experiments need snowflake-ml-python 1.19.0 or later. They also need a database and schema that already exist, because that is where run information is stored. You can create an experiment in Snowsight under AI & ML » Experiments » New Experiment by choosing a name and the database and schema for its run artifacts. In code, an ExperimentTracking(session) instance with set_experiment("my_experiment") selects the experiment that later calls write to.
The privilege model follows from where the data is stored. The role needs CREATE EXPERIMENT on the schema that will hold the run artifacts, plus USAGE (or another privilege) on the parent database and schema so it can reach that schema. Privileges for other objects don't count. CREATE DATASET covers Datasets, and CREATE STAGE covers stages. Neither one lets a role create an experiment.
A run starts with start_run(name) on the tracking instance. It returns a Run that works as a context manager. Snowflake recommends the with form because leaving the block completes the run automatically, which makes the run's scope easy to see in the code. If you start a run without with, close it yourself with end_run("my_run"). Completion matters because a completed run becomes immutable and shows as finished in Snowsight. You can't add metrics to it afterwards, so finish all logging before the with block ends.
with exp.start_run("my_run"):
# .. Train your model and log artifactsCheckpoint 1 of 9· Fill the gap
You want a run that is completed automatically, and so becomes immutable, as soon as the block exits. Which method on the ExperimentTracking instance completes this sample?
with exp. ? ("my_run"):
# .. Train your model and log artifactsstart_run returns a Run that works as a context manager, so leaving the with block completes the run. set_experiment only picks which experiment later calls write to. end_run is the manual way to close a run started without with.
Source: docs.snowflake.comCheckpoint 2 of 9· Exam question
Which privileges does a role need to create an experiment and write runs into a given schema using Experiment Tracking?
Correct answer: C — CREATE EXPERIMENT on the target schema together with USAGE on the parent database and schema that contain it
- A. Incorrect: experiment artifacts are stored by Snowflake with the experiment, so no user-managed stage is needed.
- B. Incorrect:
CREATE DATASETgoverns Dataset objects; it does not authorize creating experiments. - C. Correct: experiments are schema-level objects, so
CREATE EXPERIMENTplusUSAGEon the database and schema is required. - D. Incorrect:
CREATE MODELgoverns Model Registry objects, not experiments.
Sources1
2.Logging parameters, step metrics, artifacts and models
Run metadata comes in three kinds, and each answers a different question. Parameters are constant inputs to training, such as a learning rate, an optimizer name or a batch size. They are set once and don't change during the run. Metrics are measured at a model step, and each value is recorded against that step. This is what lets Snowsight draw a curve: if each epoch is a step, log_metric("val_loss", value, step=epoch) inside the training loop adds one point per epoch to the same metric. If you leave out step, the value is logged at step 0, and one call after training gives you a single point, not a history.
# Log model parameters with the log_param(...) or log_params(...) methods
exp.log_param("learning_rate", 0.01)
exp.log_params({"optimizer": "adam", "batch_size": 64})
# Log model metrics with the log_metric(...) or log_metrics(...) methods
exp.log_metric("loss", 0.3, step=100)
exp.log_metrics({"loss": 0.4, "accuracy": 0.8}, step=200)| Call | What it records | Where it shows up |
|---|---|---|
| log_param / log_params | Constant training inputs, one value per name | Optional parameter columns in the runs list; string columns from list_params |
| log_metric / log_metrics (step=…) | Values measured at a model step; default step is 0 | One column per metric in the runs list, metric charts, float columns from list_metrics |
| log_artifact | A local file uploaded under an artifact_path | Run artifacts, retrievable with LIST/GET on snow://experiment/… |
| log_model | A model logged to the registry from the active run | Linked model versions in the run and comparison views |
| set_live_logging_status(True) | stdout and stderr of the notebook or SPCS workload | The default event table, shown in the run's Logs tab |
The other channels have specific jobs. log_artifact('/tmp/file.txt', artifact_path='artifacts') uploads a file, but Snowsight doesn't read numbers out of it to chart them. Charts come from logged metrics only. exp.log_model(model, model_name="my_model", ...) logs the model to the registry from inside the run. You pass either signatures or sample_input_data, and the run then shows the resulting model version as linked. Live logging sends stdout and stderr from a Snowflake Notebook or another SPCS workload, such as an ML Job, to the default event table, and you read them in the run's Logs tab. It captures console output, not metric history, and it doesn't work in legacy notebooks.
For XGBoost, LightGBM and Keras you can skip manual calls and use autologging. You build a callback such as SnowflakeLightgbmCallback(exp, model_name="name", model_signature=sig), pass it to fit inside start_run, and the framework's parameter and metric updates are logged to the active run. Manual logging is still needed for models that don't support autologging and for pre-trained models.
Checkpoint 3 of 9· Check yourself
A data scientist trains a Keras-free custom model for 30 epochs inside an active run and wants Snowsight to chart validation loss as a curve with one point per epoch. Which logging approach gives that chart?
Metrics are recorded at a model step, so logging val_loss with step=epoch in the loop adds one point per epoch to the same metric. Parameters are constant inputs, not a history. A single log_metric call without step records one point at step 0. Snowsight does not chart numbers from uploaded artifact files.
“metrics are evaluated at a model step. You can choose to represent a training epoch as a corresponding step.”Source: docs.snowflake.com
Checkpoint 4 of 9· Exam question
A team trains an XGBoost model in a Container Runtime notebook and wants every epoch's validation loss stored so that the loss curve can be plotted per run in Snowsight. Which call pattern achieves this?
Correct answer: A — Call log_metric("val_loss", value, step=epoch) inside the loop so each epoch is a separate point on that metric
- A. Correct:
log_metricaccepts astepargument, so one metric name holds a series of points that Snowsight plots as a curve. - B. Incorrect: a single
log_metriccall stores one value at the default step; Snowflake does not rebuild history from the event table. - C. Incorrect: artifacts are stored files and are not parsed into charts; the metrics view only reads logged metrics.
- D. Incorrect: parameters are constant training inputs and are not step-indexed, so they cannot represent a time series.
Sources1
3.Comparing runs, retrieving artifacts and staying under the limits
Experiments are evaluated in Snowsight under AI & ML » Experiments. The runs list shows the run name, status, created date and one column per metric, and you can add parameters as extra columns. You can select up to five runs and press Compare. The comparison view puts their metadata, parameters, metrics and model version information side by side. Linked model versions come from the Model Lineage feature, which requires Enterprise Edition or above.
In code, list_metrics() and list_params() return Snowpark DataFrames with one row per run. Both include a run_name column, and each logged metric becomes a float column while each parameter becomes a string column. Because they are DataFrames, you can join metrics to parameters on run_name, filter for runs with loss under a threshold, or call to_pandas() and analyse them locally. Pass run_name="RUN_1" to get a single run's values. Artifacts are stored at a snow://experiment/ path and can be retrieved with SQL at any time, during or after a run.
LIST snow://experiment/my_experiment/versions/my_run/logs;
GET snow://experiment/my_experiment/versions/my_run/logs/log0.txt file:///tmp;Plan cleanup from the start, because each limit applies at a different level. A nightly retraining job that adds one run to the same experiment reaches 500 runs in about sixteen months. After that, new runs fail until old ones are removed. ALTER EXPERIMENT my_experiment DROP RUN my_run; removes a single run. DROP EXPERIMENT my_experiment; removes the experiment and all of its run artifacts. The metric and parameter limits apply to each run, not to the schema or the experiment.
| Limit | Scope | Cleanup when reached |
|---|---|---|
| 500 experiments | Per schema | DROP EXPERIMENT (removes all its run artifacts) |
| 500 runs | Per experiment | ALTER EXPERIMENT … DROP RUN |
| 1000 unique parameters | Per run | Log fewer distinct parameter names |
| 200 unique metrics | Per run | Log fewer distinct metric names |
Experiments have no separate charge, but they still cost something. You pay storage for run artifacts and warehouse credits for the Snowsight charts, because virtual warehouses render them. Dropping runs you no longer need keeps you under the run limit and also cuts storage.
Checkpoint 5 of 9· Check yourself
For about sixteen months, a nightly retraining job has added one run to the same experiment. Each run logs about 40 metrics. Now new runs fail to start. What is the most likely cause, and what is the fix?
Each experiment can hold at most 500 runs, and one run a night reaches that in about sixteen months. You free room by dropping individual runs. The 200-metric limit applies to each run, not to the schema. DROP EXPERIMENT would delete the experiment together with all of its run artifacts.
“Each experiment is limited to 500 runs.”Source: docs.snowflake.com
Checkpoint 6 of 9· Exam question
A team runs nightly retraining and each night adds a new run to the same experiment. After about a year, start_run begins failing. Which documented limit is the most likely cause?
Correct answer: D — The experiment has reached the cap of 500 runs, so older runs must be dropped before new runs can be created
- A. Incorrect: the 200-metric limit is per run, not per schema, and it does not block creating runs.
- B. Incorrect: 1,000 versions is a Model Registry limit per model and is unrelated to experiment runs.
- C. Incorrect: Snowflake does not document a per-role concurrent-run cap; the run count limit applies per experiment.
- D. Correct: an experiment holds at most 500 runs; clean up with
ALTER EXPERIMENT ... DROP RUNor move to a new experiment.
Sources1
4.Hyperparameter optimization with the Tuner API
The Snowflake ML HPO API runs many training trials in parallel and keeps the best configuration. It works with any model: open-source frameworks and Snowflake ML's own distributed training APIs both fit. You launch it from a Snowflake Notebook on Container Runtime. By default the trials spread across the CPU or GPU cores of a single node in the SPCS compute pool, and they can scale to multiple nodes. Choosing those resources belongs to compute pools and Container Runtime. Data reaches the trials through a dataset_map, a dictionary that maps names such as "train" and "test" to DataConnector objects.
A tuning job has three parts: a search algorithm, a search space and a configuration. GridSearch tries every combination in a grid you define. RandomSearch samples the space at random and works well when the space is large or mixes continuous and discrete values. BayesOpt uses a probabilistic model of earlier trials to choose the next configuration. The search space uses sampling functions to describe each hyperparameter's range and type.
| Function | Samples | Typical use |
|---|---|---|
| uniform(lower, upper) | A continuous value, uniformly | Dropout rates, regularization strengths |
| loguniform(lower, upper) | A value uniformly in log space | Learning rates spanning orders of magnitude |
| randint(lower, upper) | An integer, lower inclusive and upper exclusive | Number of layers, epochs or estimators |
| choice(options) | One value from a list | Categorical options such as activation functions |
TunerConfig sets the goal and the budget. metric names the value to optimize, such as accuracy. mode is "max" or "min". search_alg picks the strategy, num_trials sets how many configurations to try, and max_concurrent_trials sets how many run at once. Bayesian search needs earlier results to choose the next trial, so max_concurrent_trials=1 gives each trial the most information, at the cost of running them one after another.
The training function connects your code to the tuner. Inside it, get_tuner_context() returns the hyperparameters for the current trial and the dataset map. The function then trains and calls report with the metrics and the model. If a trial doesn't report its metrics, the search algorithm has nothing to learn from.
def train_func():
tuner_context = get_tuner_context()
config = tuner_context.get_hyper_params()
dm = tuner_context.get_dataset_map()
...
tuner_context.report(metrics={"accuracy": accuracy}, model=model)tune.Tuner(train_func, search_space, tuner_config).run(dataset_map=dataset_map) distributes the trials and returns a TunerResults object. Its results attribute is a pandas DataFrame with the metrics and configuration of every trial. best_result is the row for the best trial, and best_model is that trial's model instance, if one was reported. Because data comes in through Snowflake Datasets or DataFrames, ML lineage is captured automatically. In practice, the best trial's settings and metrics are what you log as parameters and metrics in a run, so the tuned model can be compared with the rest of the experiment.
Checkpoint 7 of 9· Fill the gap
In a BayesOpt tuning job, every trial has to return its accuracy and trained model to the tuner so the search algorithm can choose the next configuration. Which TunerContext method fills the blank?
def train_func():
tuner_context = get_tuner_context()
config = tuner_context.get_hyper_params()
dm = tuner_context.get_dataset_map()
...
tuner_context. ? (metrics={"accuracy": accuracy}, model=model)report sends the trial's metrics and model back to the tuner, and the search algorithm learns only from reported trials. log_metrics and log_params are ExperimentTracking calls that write to a run, not to the tuner. get_dataset_map only reads the trial's input data.
Source: docs.snowflake.comSources2
5.Bringing external models in with the CustomModel API
The Model Registry logs built-in model types directly. For anything else, such as a PyCaret pipeline, a pickled model from an open-source repository or a hand-written algorithm, you subclass snowflake.ml.model.custom_model.CustomModel. The subclass gets a ModelContext holding everything inference needs and exposes methods decorated with @custom_model.inference_api. Each decorated method takes a pandas DataFrame and returns one.
# Initialize ModelContext with an in-memory model object
# my_model can be any supported model type (e.g., sklearn, xgboost, lightgbm, and others)
model_context = custom_model.ModelContext(
my_model=my_model,
)
# Define a custom model class that utilizes the context
class ExampleBringYourOwnModel(custom_model.CustomModel):
def __init__(self, context: custom_model.ModelContext) -> None:
super().__init__(context)
@custom_model.inference_api
def predict(self, input: pd.DataFrame) -> pd.DataFrame:
# Use the model with key 'my_model' from the context to make predictions
model_output = self.context['my_model'].predict(input)
return pd.DataFrame({'output': model_output})ModelContext takes keyword arguments of two kinds. An in-memory object of a supported type, such as an XGBoost or scikit-learn model, can be passed directly, and Snowflake ML serializes it for you. A string file path points to a pickle, a JSON configuration or a parameter file. Your __init__ opens it from self.context['my_file_path']. Both kinds are packaged with the model. Helper code your model imports, such as a local scoring_utils/ package, is included through code_paths when you log the model.
Always reach the wrapped model through the context. If predict uses a notebook variable directly, or sets self.model = model instead of self.context['my_model'], a second copy of the model is captured in a closure. The serialized model files can then grow much larger.
Write every inference method to handle many rows. Inference methods can also accept options that control behavior, such as temperature or max_tokens. Each option must be keyword-only (declared after *), must have a type annotation (int, float, str, bool, bytes, datetime.datetime, or a list of a supported type) and must have a default value. When the model is logged, these options are added to its signature automatically. Callers can override them at inference time or leave them out and get the defaults.
Checkpoint 8 of 9· Fill the gap
This wrapper should expose predict as a method the registered model can serve. Which custom_model decorator fills the blank?
# Initialize ModelContext with an in-memory model object
# my_model can be any supported model type (e.g., sklearn, xgboost, lightgbm, and others)
model_context = custom_model.ModelContext(
my_model=my_model,
)
# Define a custom model class that utilizes the context
class ExampleBringYourOwnModel(custom_model.CustomModel):
def __init__(self, context: custom_model.ModelContext) -> None:
super().__init__(context)
@custom_model. ?
def predict(self, input: pd.DataFrame) -> pd.DataFrame:
# Use the model with key 'my_model' from the context to make predictions
model_output = self.context['my_model'].predict(input)
return pd.DataFrame({'output': model_output})A CustomModel exposes its inference methods with the @custom_model.inference_api decorator. Each decorated method takes a pandas DataFrame and returns one. ModelContext holds the files and model objects that inference needs, and model_signature is not a decorator on a CustomModel.
Source: docs.snowflake.comSources3
6.Tying a model version to its Dataset version
A run is only reproducible if you know exactly which rows trained the model. A source table that changes every day can't tell you that. Snowflake Datasets (versioned, immutable snapshots) can, because each version is a fixed point-in-time snapshot. Training from a specific Dataset version and logging the model so the registry records that link means the model version always points to data that still exists unchanged. Six months later you can retrain on exactly the same rows.
Lineage for a model is recorded when the model is logged to the Model Registry. The simplest way to get it is to call ds.read.to_snowpark_dataframe() on the selected version. That DataFrame points to the version's materialized data, not to the original query. Train with Snowpark ML on it, then log the model. Training on a Snowpark DataFrame records lineage automatically, which completes the graph from source data through feature views and the Dataset to the model.
Lineage is not automatic when you train on a pandas DataFrame, train without Snowpark ML, or train outside Snowflake. In those cases, pass a Snowpark DataFrame backed by the Dataset (or other source object) as sample_input_data to log_model, and the registry records the link. Two caveats apply. Only objects created after ML Lineage is enabled in the account carry lineage, and lineage information is not replicated.
To read lineage back, call the lineage method on a ModelVersion, Dataset or FeatureView. direction is "upstream" or "downstream" (downstream is the default), and domain_filter limits results to types such as "dataset" or "model". model_version.lineage(direction="upstream") shows where a model's training data came from. The call below runs the other way, from a Dataset down to the models trained on it. Exploring lineage from Python requires the VIEW LINEAGE account privilege, which ACCOUNTADMIN has by default and can grant to other roles. Each artifact's Snowsight page also has a Lineage tab.
my_dataset.lineage(direction="downstream", domain_filter=["model"])Checkpoint 9 of 9· Check yourself
Your team pulls a Dataset version into pandas with to_pandas(), trains a scikit-learn model on it and logs the model to the Model Registry. Working as ACCOUNTADMIN, you call model_version.lineage(direction="upstream"), and no Dataset comes back. What should the team change so the model version links to the Dataset version it was trained on?
Lineage is recorded automatically only when the model is trained on a Snowpark DataFrame. When training uses pandas, you link the model by passing a Snowpark DataFrame backed by the source object as sample_input_data to log_model. Run parameters don't create registry lineage. VIEW LINEAGE only controls who can explore lineage, and ACCOUNTADMIN already has it.
“passing a Snowpark DataFrame backed by the source data object as sample_data to the Model Registry’s log_model method”Source: docs.snowflake.com
Sources
(5)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“requires the CREATE EXPERIMENT privilege on the schema where run artifacts are stored, plus USAGE or any other privilege on the parent database”
↩︎ Experiments, runs and the privileges behind them“Completing a run makes it immutable and presents it as finished in Snowsight.”
↩︎ Experiments, runs and the privileges behind them“Snowflake Experiments require snowflake-ml-python version 1.19.0 or later.”
↩︎ Experiments, runs and the privileges behind them“You can choose to represent a training epoch as a corresponding step.”
↩︎ Logging parameters, step metrics, artifacts and models“When live logging is enabled, stdout and stderr output from the notebook is written to the Snowflake default event table.”
↩︎ Logging parameters, step metrics, artifacts and models“You can autolog training information for XGBoost, LightGBM, or Keras models during model training.”
↩︎ Logging parameters, step metrics, artifacts and models“Each experiment is limited to 500 runs.”
↩︎ Comparing runs, retrieving artifacts and staying under the limits“You can select up to five runs in your experiment.”
↩︎ Comparing runs, retrieving artifacts and staying under the limits“list_metrics includes a run_name column and one float column per logged metric”
↩︎ Comparing runs, retrieving artifacts and staying under the limits“The charts in the UI are powered by virtual warehouses.”
↩︎ Comparing runs, retrieving artifacts and staying under the limits“Viewing linked model versions is part of the Model Lineage feature, which is only available for customers on Enterprise Edition and above.”
↩︎ Comparing runs, retrieving artifacts and staying under the limits“Each experiment consists of a series of runs, which are metadata and artifacts from your training.”
↩︎ Key concept“Parameters are constant inputs to the training model, while metrics are evaluated at a model step.”
↩︎ Exam trap 1“Runs are limited to 1000 unique parameters and 200 unique metrics.”
↩︎ Exam trap 2“metrics are evaluated at a model step. You can choose to represent a training epoch as a corresponding step.”
↩︎ Checkpoint - 2.
“The workload scales out to the CPU or GPU cores that are available on a single node in the SPCS compute pool.”
↩︎ Hyperparameter optimization with the Tuner API“The Snowflake ML HPO API requires the training metrics and hyperparameters from each training run to optimize the hyperparameters effectively.”
↩︎ Hyperparameter optimization with the Tuner API“results: A Pandas DataFrame containing metrics and configurations for every trial.”
↩︎ Hyperparameter optimization with the Tuner API“efficient data ingestion via Snowflake Datasets or Dataframes and automatic ML lineage capture”
↩︎ Hyperparameter optimization with the Tuner API“The HPO API evaluates every possible combination of hyperparameters.”
↩︎ Prediction - 3.https://docs.snowflake.com/en/developer-guide/snowflake-ml/model-registry/bring-your-own-model-typesOfficial docs
“Serializable models trained using external tools or obtained from open source repositories can be used with CustomModel.”
↩︎ Bringing external models in with the CustomModel API“Files can be serialized models, configuration files, or files containing parameters.”
↩︎ Bringing external models in with the CustomModel API“When this model is logged, the parameters are automatically included in the model signature.”
↩︎ Bringing external models in with the CustomModel API“Include additional code with your model using code_paths.”
↩︎ Bringing external models in with the CustomModel API“Accessing the model directly captures a second copy of the model in a closure, which results in significantly larger model files during serialization.”
↩︎ Exam trap 3“Don’t assume that the input DataFrame will always contain a single row.”
↩︎ Prediction - 4.
“Each version is an immutable, point-in-time snapshot of the data managed by the Dataset.”
↩︎ Tying a model version to its Dataset version“The trained model can then be logged to Snowflake Model Registry, automatically completing the ML Lineage graph linking source data, feature views, datasets”
↩︎ Tying a model version to its Dataset version - 5.
“Lineage for models is captured when the model is logged to the Model Registry.”
↩︎ Tying a model version to its Dataset version“Training a model using Snowpark ML automatically generates lineage records if the model is trained from a Snowpark DataFrame.”
↩︎ Tying a model version to its Dataset version“Users need the VIEW LINEAGE privilege to explore lineage from Python APIs.”
↩︎ Tying a model version to its Dataset version“Only objects created after the ML Lineage feature is enabled in your account contain lineage information.”
↩︎ Tying a model version to its Dataset version“passing a Snowpark DataFrame backed by the source data object as sample_data to the Model Registry’s log_model method”
↩︎ Exam trap 4“passing a Snowpark DataFrame backed by the source data object as sample_data to the Model Registry’s log_model method”
↩︎ Checkpoint