Snowflake ML Experiments: Logging and Comparing Training Runs

1.How an experiment is structured: experiment, runs, run metadata

Snowflake ML Experiments give you an organized way to evaluate training results. You can compare the effect of hyperparameter changes, different target metrics, or different model types, and then pick the best model. The structure has levels. At the top is the experiment, a schema-scoped object. Inside it are runs, and each run holds its own parameters, metrics and artifacts. Snowflake does not restrict what you store as an artifact: anything that helps you evaluate the model is allowed.

A note on the exam guide's phrase "nested experiments": the sources for this lesson describe only this experiment → run → metadata hierarchy. You can also group artifacts into sub-paths inside a run (the artifact_path argument, covered in the next section). The sources do not describe parent/child runs or any nesting flag, so don't assume one exists.

Before you start, check the prerequisites. Experiments need snowflake-ml-python 1.19.0 or later. Your role needs the CREATE EXPERIMENT privilege on the schema that will store run artifacts, plus a privilege such as USAGE on the parent database and schema. You can create an experiment in Snowsight under AI & ML » Experiments, where you choose a name and the database and schema for its artifacts. In Python, you start a run with start_run(name) on your ExperimentTracking instance. This returns a Run that works as a context manager. Snowflake recommends the with form because the run is completed cleanly and its scope is easy to see.

with exp.start_run("my_run"):
  # .. Train your model and log artifacts

Sources1

2.Logging parameters, metrics, models and artifacts

You can fill a run in two ways. For XGBoost, LightGBM and Keras, use autologging. You register a Snowflake callback that points at your experiment, and every parameter or metric change during training is logged to the active run. The callback also accepts a model name and a model signature, so the trained model can be logged as well.

from lightgbm import LGBMClassifier

from snowflake.ml.experiment.callback.lightgbm import SnowflakeLightgbmCallback
from snowflake.ml.model.model_signature import infer_signature

sig = infer_signature(X, y)
callback = SnowflakeLightgbmCallback(
    exp, model_name="name", model_signature=sig
)
model = LGBMClassifier()
with exp.start_run("my_run"):
    model.fit(X, y, eval_set=[(X, y)], callbacks=[callback])

Some models don't support autologging, and pre-trained models have no training loop to hook into. For these, log by hand while a run is active. Parameters and metrics differ in one way. A parameter is a constant input to training. A metric is measured at a particular step, so you can treat each epoch as a step. If you don't pass a step, it defaults to 0. log_model registers the model in the experiment's model registry, and the model version is then linked to the run. log_artifact uploads a local file, optionally under an artifact_path sub-folder of the run.

exp.log_metric("loss", 0.3, step=100)
exp. ? ({"loss": 0.4, "accuracy": 0.8}, step=200)

When a run is active in a Snowflake Notebook or another SPCS workload, such as an ML Job, you can also capture console output. Turn on live logging, and stdout and stderr are written to the default event table. You can read them on the run's Logs tab in the Experiments UI. This does not work in legacy notebooks.

experiment.set_live_logging_status(True)

Sources1

3.Completing, comparing and cleaning up runs

A run ends in one of two ways. If you used a with block, it completes when the block exits. Otherwise, call end_run("my_run"). A completed run is immutable and shows as finished in Snowsight. This is what makes a run trustworthy as a record of a training attempt.

To monitor an experiment, open it under AI & ML » Experiments. The runs list shows the name, status, created date and one column per metric, and you can add parameters as extra columns. From a run, you can open its artifacts, metric charts and linked model versions. Viewing linked model versions belongs to the Model Lineage feature, which needs Enterprise Edition or higher. You can select up to five runs and choose Compare to see their metadata, parameters, metrics and model versions side by side.

To compare runs in code, use list_metrics() and list_params(). Each returns a Snowpark DataFrame with one row per run. list_metrics has a run_name column and one float column per metric. list_params has a run_name column and one string column per parameter. You can join and filter them with Snowpark, or convert them to pandas. Pass run_name= to get the values for a single run. To read artifacts, use SQL against the experiment's snow:// path, during or after the run:

LIST snow://experiment/my_experiment/versions/my_run/logs;
GET snow://experiment/my_experiment/versions/my_run/logs/log0.txt file:///tmp;

You can also monitor an experiment from SQL, without opening Snowsight. SHOW RUNS IN EXPERIMENT lists the runs. Its metadata column is a JSON object holding each run's status (RUNNING or FINISHED) and its metrics. Only the latest metric value, the one with the highest step, is included, so use list_metrics or the run view when you need the full metric history. The command needs USAGE on the experiment and does not need a running warehouse.

SHOW RUNS [ LIKE '<pattern>' ] IN EXPERIMENT <name>

Monitoring across the model development lifecycle also means knowing where a model came from. Lineage for a model is captured when the model is logged to the Model Registry. A model trained from a Snowpark DataFrame gets lineage records automatically. For a model trained another way, pass a Snowpark DataFrame backed by the source data object as sample_input_data to log_model. You can then call model_version.lineage(direction="upstream") to see where its training data came from. Lineage tracing needs the VIEW LINEAGE privilege. Together with the runs that linked the model version, this ties each model back to the data and the training attempt that produced it.

DROP EXPERIMENT my_experiment removes an experiment and all of its run artifacts. ALTER EXPERIMENT my_experiment DROP RUN my_run removes a single run. Experiments have no extra charge. You pay standard costs for storing artifacts and for the warehouses that render the UI charts. There are also fixed limits:

Sources123

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.After a run finishes, you can keep adding corrected metrics to it.Why is that wrong?

   Completing a run, either with end_run or by leaving the with block, makes it immutable. New measurements belong in a new run.

   Covered in Completing, comparing and cleaning up runs

2. 2.SHOW RUNS IN EXPERIMENT returns the full step-by-step history of every metric.Why is that wrong?

   The metadata column reports only the latest value of each metric, the one with the highest step. Use list_metrics or the run view for more.

   Covered in Completing, comparing and cleaning up runs

3. 3.Live logging of stdout/stderr works in any Snowflake notebook.Why is that wrong?

   Live logging is for active runs in Snowflake Notebooks or other SPCS workloads such as ML Jobs. It does not work in legacy notebooks.

   Covered in Logging parameters, metrics, models and artifacts