What you will be able to do
- Create a model version monitor with valid timestamp, window, prediction, actual and baseline settings
- Explain why a monitor created without a baseline cannot report drift, and how to fix it
- Query drift and performance metrics with MODEL_MONITOR_DRIFT_METRIC and MODEL_MONITOR_PERFORMANCE_METRIC and turn them into threshold checks
- Diagnose a monitor that has stopped refreshing
- Choose between a model monitor's drift metrics, Anomaly Detection functions and data metric function anomaly detection for tracking data drift and statistical anomalies
Key concept
Model version monitor — A schema object tied to exactly one Model Registry model version. It reads a source table of stored inferences, aggregates them into day-sized windows and, if it was given a baseline snapshot at creation, compares production data against that snapshot to measure drift.
1.What a model version monitor watches
A model can pass every test on release day and still decay in production. Inputs drift, training assumptions go stale and pipelines break upstream, and none of this throws an error. ML Observability catches that decay by tracking deployed Model Registry models across performance, drift and volume. It also works when inference ran outside Snowflake, because it reads stored inference data, not live calls.
The raw material is a monitoring log: a table holding an ID, a timestamp, the features, the predictions and a ground-truth label. The monitor refreshes itself by querying that source and updates its reports from the result. It supports regression, binary classification and multi-class classification models.
Monitors cannot be shared. If version v1 and version v2 both serve traffic during a rollout, you need two monitors, and you compare them side by side in the dashboard. Creating one needs the CREATE MODEL MONITOR privilege on the target schema, and an account can hold at most 250 monitors. A second kind, the gateway model monitor, covers real-time inference behind a Snowflake Gateway, including A/B tests. Everything else on this page is about version monitors unless it says otherwise.
Drift and statistical anomalies are different signals, and Snowflake tracks them in different places. Every version monitor produces drift metrics (distribution changes or data shifts) and statistical metrics (counts and null values), so a feature whose distribution moves away from the baseline, or whose null count jumps, shows up without any extra setup. An anomaly is a narrower question: is today's value outside the range its own history predicts? For a time series such as daily row volume or a feature's null rate, the Anomaly Detection ML function answers it. Train it on history, call DETECT_ANOMALIES on recent data, and wrap that call in a stored procedure that an alert or task runs on a schedule, keeping rows where is_anomaly is true. The alert delivery itself is covered in the alerting pages. The data metric function route is the zero-model option: Snowflake flags a DMF result that falls above or below an expected range learned from its history, and sends a notification enabled at the database level. Pick a monitor drift metric when the question is whether production differs from a fixed reference, and anomaly detection when the question is whether a value broke from its own recent pattern.
Checkpoint 1 of 8· Check yourself
During a rollout, versions v1 and v2 of a Model Registry model both serve traffic. v1 runs inference in Snowflake. v2 runs inference on an external service, and its predictions and labels are written back to a Snowflake table. You want to compare drift and performance for both versions in ML Observability. What do you set up?
A monitor covers exactly one model version and cannot be shared, so the two versions need two monitors, which you then compare side by side in the dashboard. Monitors read stored inference data, so v2's external inference is covered once its results are in Snowflake. Gateway monitors are for real-time inference behind a Snowflake Gateway, not for external inference.
“Each model version can have exactly one monitor, and each monitor can monitor exactly one model version; they cannot be shared.”Source: docs.snowflake.com
2.Creating a monitor: source, timestamp and window settings
CREATE MODEL MONITOR my_monitor WITH
MODEL = my_model
VERSION = 'v1'
FUNCTION = 'predict'
SOURCE = mydb.myschema.scoring_data
WAREHOUSE = compute_wh
REFRESH_INTERVAL = '1 day'
AGGREGATION_WINDOW = '1 day'
TIMESTAMP_COLUMN = event_time
PREDICTION_SCORE_COLUMNS = ( 'prediction_score' )
ACTUAL_SCORE_COLUMNS = ( 'actual_score' );Two time settings in this statement look alike but do different jobs. REFRESH_INTERVAL sets how often the monitor refreshes its internal state. It accepts seconds, minutes, hours or days, with a floor of 60 seconds. AGGREGATION_WINDOW sets the size of the buckets that metrics are computed over, and on a version monitor it accepts whole days only, with a minimum of one day. Hour-sized windows exist only on gateway monitors. So a version monitor declared with AGGREGATION_WINDOW = '6 hours' is rejected, whatever the refresh interval is. If predictions arrive hourly, they are still summarised per day.
The timestamp column has a strict type. TIMESTAMP_COLUMN must be TIMESTAMP_NTZ, so a source that stores TIMESTAMP_TZ or text timestamps needs a view that casts the column first. Prediction and actual columns follow the model task. Binary classification accepts either scores or classes as predictions, but actuals must be classes. Multi-class uses classes on both sides, and regression uses numbers on both sides.
| Parameter | Rule |
|---|---|
| TIMESTAMP_COLUMN | Must be of type TIMESTAMP_NTZ |
| AGGREGATION_WINDOW | Days only on version monitors; hours are allowed only on gateway monitors |
| REFRESH_INTERVAL | Minimum '60 seconds'; seconds, minutes, hours or days |
| PREDICTION_SCORE_COLUMNS / PREDICTION_CLASS_COLUMNS | At least one prediction column is mandatory |
| ACTUAL_SCORE_COLUMNS / ACTUAL_CLASS_COLUMNS | Optional, but without them accuracy metrics are not computed |
| SEGMENT_COLUMNS | STRING columns, at most 5 per monitor |
| CUSTOM_METRIC_COLUMNS | NUMBER columns tracked next to the model, not treated as features |
Two optional settings widen what the monitor tracks. CUSTOM_METRIC_COLUMNS covers numeric business values such as latency_ms or num_impressions that belong next to model quality but are not model inputs. SEGMENT_COLUMNS splits every metric by a string column such as region or customer tier. To segment on a numeric column, bucket it into categories first. Gateway monitors join labels in a different way: ID_COLUMNS and GROUND_TRUTH must be given together, so a ground-truth table supplied without ID columns produces no accuracy metrics.
Checkpoint 2 of 8· Check yourself
A team's scoring table gets new predictions every hour. They want drift and accuracy for one Model Registry version summarised in 6-hour buckets, so their CREATE MODEL MONITOR for that version sets REFRESH_INTERVAL = '1 hour' and AGGREGATION_WINDOW = '6 hours'. What happens?
AGGREGATION_WINDOW sets the bucket size for metrics, and on a version monitor it must be whole days, no matter what the refresh interval is. REFRESH_INTERVAL is a separate setting: it accepts hours, and its minimum is 60 seconds.
“For model version monitors, only days are supported. For gateway model monitors, both hours and days are supported.”Source: docs.snowflake.com
Checkpoint 3 of 8· Exam question
Which column type must the TIMESTAMP_COLUMN of a model version monitor's source table use?
Correct answer: B — TIMESTAMP_NTZ, because the monitor buckets records into windows with timezone-naive values.
- A. Incorrect: TIMESTAMP_TZ values carry an offset, but the monitor does not accept that type for its timestamp column.
- B. Correct: the timestamp column of a model version monitor must be TIMESTAMP_NTZ.
- C. Incorrect: TIMESTAMP_LTZ depends on session timezone and is not the supported type for the monitor timestamp column.
- D. Incorrect: the monitor does not parse text; a string column would be rejected at creation time.
Checkpoint 4 of 8· Exam question
A model version monitor is being created for a model whose predictions arrive hourly. The engineer sets AGGREGATION_WINDOW = '6 hours' to get finer trend lines. What is the result?
Correct answer: A — Creation fails, because a model version monitor supports only whole-day aggregation windows, minimum 1 day.
- A. Correct: for model version monitors the aggregation window is expressed in days with a 1-day minimum; hour windows are available only for gateway monitors.
- B. Incorrect: segment columns split metrics by category and do not unlock sub-day windows.
- C. Incorrect: warehouse size has no effect on window rounding; an unsupported window is an error, not an adjustment.
- D. Incorrect: matching REFRESH_INTERVAL does not change the whole-day requirement for the aggregation window of a version monitor.
3.Baselines: the reference that drift is measured against
Drift is a comparison, and BASELINE supplies the reference side. It names a table holding a snapshot of data shaped like SOURCE, typically the features and predictions the model was validated on. At creation, Snowflake embeds a snapshot of that table inside the monitor object. Changing the training table afterwards therefore does not move the reference.
The parameter is optional, and leaving it out is the most common way to get a monitor with no drift. Volume and performance still work, because they need only the source data. Drift queries fail, since a version monitor must have a baseline before any drift metric is computed. A monitor's basic configuration cannot be changed after it is created. To add a baseline, drop the monitor and create it again with BASELINE set. A flag column in the source does not create a reference distribution, and a grant cannot make the monitor discover one.
Checkpoint 5 of 8· Check yourself
Consider a regression model version monitor that was created with prediction and actual score columns but no BASELINE. An engineer queries MODEL_MONITOR_PERFORMANCE_METRIC for 'RMSE' and MODEL_MONITOR_DRIFT_METRIC for 'WASSERSTEIN' on a feature column. What should they expect?
Volume and performance metrics need only the source data, so RMSE works. Drift compares production data against a reference, so on a version monitor with no baseline the drift query errors. There is no fallback reference to compare against.
“The model version monitor must have a baseline set for the drift metric to be computed.”Source: docs.snowflake.com
Checkpoint 6 of 8· Exam question
A team created a model version monitor without a BASELINE table. The dashboard shows volume and performance metrics but no drift. They now want drift metrics on this same monitor. What must they do?
Correct answer: A — Drop the monitor and recreate it with BASELINE set to a snapshot table of training features and predictions.
- A. Correct: drift needs baseline data, and the baseline cannot be added to an existing monitor, so the monitor must be dropped and recreated with a baseline table.
- B. Incorrect: the basic configuration of a model monitor, including its baseline, cannot be changed after creation, so there is no property to set with ALTER.
- C. Incorrect: the monitor does not infer a baseline from a flag column or from the earliest window; the baseline is a separate table passed at creation.
- D. Incorrect: privileges on a training table do not cause automatic baseline discovery; the baseline must be named explicitly when the monitor is created.
4.Querying drift and performance metrics for thresholds
The dashboard is for people. Threshold monitoring uses two table functions that return the same numbers as rows you can filter. MODEL_MONITOR_DRIFT_METRIC takes a monitor, a metric and a column. The column can be any feature, prediction or actual column. The metrics are JENSEN_SHANNON, DIFFERENCE_OF_MEANS, WASSERSTEIN and POPULATION_STABILITY_INDEX. Asking for a numeric drift metric on a non-numeric feature is an error.
SELECT * FROM TABLE(MODEL_MONITOR_DRIFT_METRIC(
'MY_MONITOR', 'JENSEN_SHANNON', 'MODEL_PREDICTION', '1 DAY', DATEADD('DAY', -30, CURRENT_DATE()), CURRENT_DATE())
)MODEL_MONITOR_PERFORMANCE_METRIC works the same way, but the metrics available depend on the model task and on which columns the monitor was given. Each call returns one row per window, with EVENT_TIMESTAMP, METRIC_VALUE and the record counts COUNT_USED and COUNT_UNUSED. Some metrics also return a CI_VALUE confidence interval. A threshold check is an ordinary predicate such as METRIC_VALUE < 0.8 on the result. A sudden fall in COUNT_USED warns of a volume problem before accuracy moves.
SELECT * FROM TABLE(MODEL_MONITOR_PERFORMANCE_METRIC(
'MY_MONITOR', 'RMSE', '1 DAY', DATEADD('DAY', -30, CURRENT_DATE()), CURRENT_DATE())
)| Model task | Metrics | Required columns |
|---|---|---|
| Regression | RMSE, MAE, MAPE | prediction_score and actual_score |
| Binary classification | ROC_AUC | prediction_score and actual_class |
| Binary classification | CLASSIFICATION_ACCURACY, PRECISION, RECALL, F1_SCORE | prediction_class and actual_class |
| Multi-class classification | CLASSIFICATION_ACCURACY, MACRO_AVERAGE_PRECISION, MACRO_AVERAGE_RECALL, MICRO_AVERAGE_PRECISION, MICRO_AVERAGE_RECALL | prediction_class and actual_class |
Three defaults affect threshold queries. With no explicit range, a version monitor looks back 60 days at a granularity of 1 DAY, so a check that omits the range keeps rescanning old windows. A segment is queried by passing extra_args JSON, and only one column:value pair is allowed per call. In production queries, name the columns you need instead of using *, so that a schema change does not break the check.
Checkpoint 7 of 8· Fill the gap
This query feeds a daily threshold check for a regression model version monitor. The monitor was created with PREDICTION_SCORE_COLUMNS and ACTUAL_SCORE_COLUMNS. Which metric name completes the call so it returns a value for each window?
SELECT * FROM TABLE(MODEL_MONITOR_PERFORMANCE_METRIC(
'MY_MONITOR', ' ? ', '1 DAY', DATEADD('DAY', -30, CURRENT_DATE()), CURRENT_DATE())
)Which performance metrics you can use depends on the model task and the monitor's columns. On a regression monitor with prediction_score and actual_score, RMSE (like MAE and MAPE) is available. CLASSIFICATION_ACCURACY and ROC_AUC are classification metrics that need class columns. WASSERSTEIN is a drift metric, so it belongs to MODEL_MONITOR_DRIFT_METRIC, not this function.
Source: docs.snowflake.com5.Keeping the monitor itself healthy
A broken monitor looks just like a healthy model: the charts simply stop moving. After five consecutive refresh failures caused by its source tables, for example a dropped or renamed column, a monitor suspends its refreshes automatically. DESCRIBE MODEL MONITOR shows the cause. aggregation_status contains SUSPENDED for each stalled aggregation, and aggregation_last_error holds the SQL error behind it. Fix the source, then run ALTER MODEL MONITOR ... RESUME. The same SUSPEND and RESUME pair pauses monitoring on purpose, for example during a planned backfill.
For a visual check, open AI & ML » Models in Snowsight, choose the model and open a monitor from its Monitors list. The dashboard's graphs depend on the model task. You can change the time range, choose which metrics appear, and compare two version monitors, such as a champion and a challenger.
Checkpoint 8 of 8· Put it in order
Someone renamed a feature column in the SOURCE table of a model version monitor. Its dashboard charts have stopped updating for several days. Put the recovery steps in order.
- 1.Run ALTER MODEL MONITOR ... RESUME to restart refreshes
- 2.Read aggregation_last_error to find the SQL error that caused the suspension
- 3.Run DESCRIBE MODEL MONITOR and confirm that aggregation_status shows SUSPENDED
- 4.Fix the source table so that the column the monitor expects exists again
After five consecutive refresh failures caused by its source tables, the monitor suspends itself. DESCRIBE MODEL MONITOR shows that it is suspended and gives the error. RESUME comes only after the root cause is fixed. Resuming earlier would only lead to more failures. Dropping and recreating the monitor is not needed.
“After resolving the root cause of the refresh failure, resume the monitor by issuing ALTER MODEL MONITOR … RESUME.”Source: docs.snowflake.com
Sources1
Sources
(6)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.https://docs.snowflake.com/en/developer-guide/snowflake-ml/model-registry/model-observabilityOfficial docs
“track the quality of production models you have deployed via the Snowflake Model Registry across multiple dimensions, such as performance, drift, and volume.”
↩︎ What a model version monitor watches“The model monitor supports regression, binary classification, and multi-class classification models.”
↩︎ What a model version monitor watches“You can create a maximum of 250 model monitors per account.”
↩︎ What a model version monitor watches“Statistical metrics: Counts or null values”
↩︎ What a model version monitor watches“The minimum time granularity at which data is stored (aggregation window), currently 1 day minimum.”
↩︎ Creating a monitor: source, timestamp and window settings“Model monitors automatically suspend refreshes when they encounter five consecutive refresh failures related to the source tables.”
↩︎ Keeping the monitor itself healthy“Each model version can have exactly one monitor, and each monitor can monitor exactly one model version; they cannot be shared.”
↩︎ Key concept“Each model version can have exactly one monitor, and each monitor can monitor exactly one model version; they cannot be shared.”
↩︎ Checkpoint“After resolving the root cause of the refresh failure, resume the monitor by issuing ALTER MODEL MONITOR … RESUME.”
↩︎ Checkpoint - 2.
“monitoring your data for anomalies, by using Anomaly Detection functions within Snowflake Tasks or Alerts”
↩︎ What a model version monitor watches - 3.
“An anomaly occurs when the value returned by a DMF is above or below an expected range based on historical data.”
↩︎ What a model version monitor watches - 4.
“Name of the column in the source data that contains the timestamps. Must be of type TIMESTAMP_NTZ.”
↩︎ Creating a monitor: source, timestamp and window settings“The minimum refresh interval is '60 seconds'.”
↩︎ Creating a monitor: source, timestamp and window settings“Actual columns are optional, but accuracy metrics are not computed if they are not specified.”
↩︎ Creating a monitor: source, timestamp and window settings“For gateway model monitors, specify ID_COLUMNS and GROUND_TRUTH together to enable performance metrics monitoring, or omit both.”
↩︎ Creating a monitor: source, timestamp and window settings“These columns are not treated as features.”
↩︎ Creating a monitor: source, timestamp and window settings“A snapshot of this data is embedded within the monitor object.”
↩︎ Baselines: the reference that drift is measured against“cannot be changed after the monitor is created. You can modify only a few options using ALTER MODEL MONITOR.”
↩︎ Exam trap 1“For model version monitors, only days are supported. For gateway model monitors, both hours and days are supported.”
↩︎ Exam trap 2“For model version monitors, only days are supported. For gateway model monitors, both hours and days are supported.”
↩︎ Checkpoint“Although this parameter is optional, if it is not set, the monitor cannot detect drift.”
↩︎ Prediction - 5.
“The model version monitor must have a baseline set for the drift metric to be computed.”
↩︎ Baselines: the reference that drift is measured against“Request a numerical drift metric for a non-numeric feature.”
↩︎ Querying drift and performance metrics for thresholds“Currently, segment queries support only 1 segment column:value pair per query.”
↩︎ Querying drift and performance metrics for thresholds - 6.
“ROC_AUC: Requires the prediction_score and actual_class columns”
↩︎ Querying drift and performance metrics for thresholds