Model Interpretation in Snowflake: SHAP, Feature Importance, Dependence and Intervals

1.Why interpret a model, and what a Shapley value says

A trained model learns relationships from data instead of having them written down in advance. That is what makes machine learning useful, and it also means the result can behave like a black box. When a model underperforms, it is hard to tell why. A black box can also hide implicit biases. Regulated industries such as finance and healthcare may need evidence that a model reaches the right answers for the right reasons. To deal with this, the Snowflake Model Registry has an explainability function built on Shapley values. For each feature, a Shapley value is that feature's marginal contribution to one prediction, averaged over all possible combinations of features. The method is computationally expensive, and Snowflake presents it as the basis for interpreting and debugging models.

There are two things to notice in the table. First, contributions have a sign. Not allowing pets makes the house less desirable, so that feature pulls the price down by $25,000. Second, the reference point is not zero. It is the average prediction, and that average is calculated from background data, meaning a representative sample of the whole dataset. Background data is used throughout the rest of this lesson. It decides whether explainability can be switched on at all, and whether SHAP values from two runs can be compared.

Sources1

2.Enabling explainability when you log a model

Explainability is a property of a model version in the registry, so it is set up when you log the model. The preview supports Python-native XGBoost, CatBoost, LightGBM and scikit-learn models, plus the XGBoost, LightGBM and scikit-learn classes from snowflake.ml.modeling. It is on by default for these models when they are logged with Snowpark ML 1.6.2 or later, and the implementation uses the SHAP library. You supply background data by passing up to 1,000 rows in sample_input_data. Some tree-based models encode background data in their structure during training and may not need it supplied explicitly. Most models do need it, and every model, tree-based ones included, gets more accurate explanations when you provide it.

mv = reg.log_model(
    catboost_model,
    model_name="diamond_catboost_explain_enabled",
    version_name="explain_v0",
    conda_dependencies=["snowflake-ml-python"],
    sample_input_data = xs, # xs will be used as background data
)

The Snowpark ML 1.6.2 row matters most on the exam. Model versions are immutable, so upgrading the library does not add an explain method to a version that already exists. You load the model's Python object with ModelVersion.load and log it again as a new version, passing the background data. The environment you load into must match the deployment environment exactly, including the Python version and every library version.

mv_old = reg.get_model("model_without_explain_enabled").default
model = mv_old. ? ()
mv_new = reg.log_model(
    model,
    model_name="model_with_explain_enabled",
    version_name="explain_v0",
    conda_dependencies=["snowflake-ml-python"],
    sample_input_data = xs
)

Sources1

3.Calling explain from Python and SQL

An explainable model version has a method named explain that returns a Shapley value for each feature. A Shapley value explains one prediction made from specific inputs, so explain does nothing on its own. You pass it the rows whose predictions you want explained. In Python you call it with ModelVersion.run and the function name.

reg = Registry(...)
mv = reg.get_model("diamond_catboost_explain_enabled").default
explanations = mv.run(input_data, function_name="explain")

In SQL, the same method is exposed as a table function on a model alias. Because the explanation runs in SQL, it can be applied to every row of a table. That is useful when you need per-row reasons for a whole batch of predictions.

WITH MV_ALIAS AS MODEL DATABASE.SCHEMA.DIAMOND_CATBOOST_EXPLAIN_ENABLED VERSION EXPLAIN_V0
SELECT *
      FROM DATABASE.SCHEMA.DIAMOND_DATA,
          TABLE(MV_ALIAS!EXPLAIN(CUT, COLOR, CLARITY, CARAT, DEPTH, TABLE_PCT, X, Y, Z));

To compute a Shapley value, the algorithm perturbs input features and replaces them with background data. The output is therefore a deviation from that background. If you compare SHAP values from two datasets, for example this month's scored rows against last month's, both must have been explained against the same background data. Otherwise the baselines differ and the numbers are not comparable. One practical issue also appears in the docs: with snowflake-ml-python older than 1.7.0, XGBoost models can fail with a UnicodeDecodeError. This is caused by an incompatibility between SHAP 0.42.1 and XGBoost 2.1.1. The fix is to upgrade. If you cannot, pin XGBoost to 2.0.3 and log with relax_version set to False.

Sources1

4.Feature impact: one prediction, the whole model, and ML Functions scores

The output of explain is a table of SHAP values. Snowflake provides three visualization functions in snowflake.ml.monitoring.explain_visualize for interpreting it, and they answer different questions.

A force plot covers one row. Each feature appears as an arrow that pushes the prediction up from the base value, shown in red, or down, shown in blue. Arrow size shows magnitude. The base_value argument defaults to 0.0, but the docs say it should normally be set to the model's mean prediction. contribution_threshold, which defaults to 0.05, hides features whose share of total absolute SHAP is too small. If no feature meets the threshold, or the threshold is outside 0 to 1, the function raises a SnowflakeMLException.

A violin plot covers many rows. It shows the distribution of SHAP values for each feature and sorts features by their absolute mean SHAP value, which gives a global ranking of feature impact.

Snowflake ML Functions report feature impact separately, through SQL methods on the model object. A classification model has SHOW_FEATURE_IMPORTANCE, and forecasting and anomaly-detection models have EXPLAIN_FEATURE_IMPORTANCE. Each returns a rank and a score between 0 and 1 for every feature. For classification, the score counts how often the model's trees used a feature to make a decision, normalized so the scores sum to 1. You cannot choose the technique. Features that are very similar to each other can split importance between them, so each looks less important than it is.

Both methods also return a feature type column, but it is not part of FORECAST output. In EXPLAIN_FEATURE_IMPORTANCE, FEATURE_TYPE separates user_provided features from those derived_from_timestamp or derived_from_endogenous, and aggregated_endogenous_features groups every feature built from the target variable. In SHOW_FEATURE_IMPORTANCE for classification, feature_type is currently always user_provided.

Sources2345

5.Partial dependence: how a feature's value shapes predictions

The violin plot shows how much a feature matters. It does not show which values of the feature push predictions up and which push them down. plot_influence_sensitivity fills that gap. It draws a SHAP dependence scatter, with the feature's values on the x-axis and the matching SHAP values on the y-axis, one point per row. From the shape of the scatter you can read the trend, the strength and direction of the influence, and any clusters that point to interactions with other features. There is also an environment difference. In Snowflake Notebooks you can pass 2D arrays covering many features and pick one from an interactive dropdown. In a local notebook you must pass one feature's SHAP values and feature values.

Sources6

6.Confidence intervals: quantifying uncertainty in predictions

Snowflake's FORECAST method returns uncertainty by default. Every forecast row has a FORECAST value plus LOWER_BOUND and UPPER_BOUND columns, which mark the edges of a prediction interval. The width is set by the prediction_interval key in the optional CONFIG_OBJECT. It must be at least 0.0 and less than 1.0, and it defaults to 0.95. A wider setting, such as 0.99, produces wider bounds because the interval has to cover more future points. A wide interval tells whoever uses the forecast that the point value is less certain.

Sources7

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.Upgrading snowflake-ml-python will add the explain method to a model version logged with an older release.Why is that wrong?

   Model versions are immutable. You load the old model and log it as a new version, passing background data in sample_input_data.

   Covered in Enabling explainability when you log a model

2. 2.Background data is optional for every model, so explainability will always work without it.Why is that wrong?

   Some model types need explicit background data, and without it explainability cannot be enabled. Even tree-based models get more accurate explanations when it is supplied.

   Covered in Enabling explainability when you log a model

3. 3.Feature importance scores from SHOW_FEATURE_IMPORTANCE are exact measures of each feature's effect.Why is that wrong?

   The scores are normalized split counts and give an approximate ranking. Very similar features can split importance between them, and the values should be treated as estimates.

   Covered in Feature impact: one prediction, the whole model, and ML Functions scores

4. 4.A force plot's default base value is the model's average prediction.Why is that wrong?

   base_value defaults to 0.0. You should normally set it to the model's mean prediction yourself.

   Covered in Feature impact: one prediction, the whole model, and ML Functions scores

5. 5.SHOW_FEATURE_IMPORTANCE scores are SHAP values, so they explain why one specific prediction was made.Why is that wrong?

   They count how often the model's trees used each feature, normalized to sum to 1. That is a model-level approximate ranking, not a per-prediction contribution measured against background data.

   Covered in Feature impact: one prediction, the whole model, and ML Functions scores

6. 6.Setting prediction_interval to 1.0 gives the widest possible forecast bounds.Why is that wrong?

   The value must be at least 0.0 and strictly less than 1.0, so 1.0 is not allowed. Values closer to 1.0, such as 0.99, give wider bounds.

   Covered in Confidence intervals: quantifying uncertainty in predictions