ML Cost Attribution in Snowflake: Object Tags, Query Tags and Feature View Costs

1.Where ML credits come from

Before you can attribute ML cost, you need to know which meters run. An ML workload in Snowflake usually uses more than one kind of compute. Feature engineering and training stored procedures run on virtual warehouses. Model serving and GPU training often run in Snowpark Container Services on compute pools. Some features run on Snowflake-managed serverless compute, and the cloud services layer coordinates all of it. Snowflake's compute cost model names these four categories separately, and each one shows up in different usage views.

Cloud services usage is only billed when it exceeds 10% of that day's warehouse usage. Serverless compute does not count toward this adjustment. In practice, most attributable ML spend sits in warehouses and compute pools, so the rest of this lesson focuses on those two.

Tracking the cost of ML features at a granular level means splitting a feature's spend into the meters it actually uses, instead of reading one lump of warehouse credits. Features served online report under their own service types in the usage views, separate from ordinary warehouse metering, so you can read each one on its own.

For the offline side, a managed feature view refreshes on a warehouse, so the most direct way to attribute its cost is to give it a dedicated warehouse. Every credit on that warehouse then belongs to the refreshes, and you can tag the warehouse with a cost center as shown in the next section. To see the cost of one dynamic table behind a feature view, query DYNAMIC_TABLE_REFRESH_HISTORY and join it with QUERY_HISTORY. Together, the service-type split for online serving and the dedicated, tagged warehouse for offline refreshes give per-feature, per-team figures.

Sources123

2.Object tags on dedicated ML resources

The simplest case is a resource used by only one team, for example a warehouse that only the fraud-model team uses. You tag the warehouse, and all of its credits go to that team. The recommended setup starts with a dedicated database and schema for tags. You then create a tag whose ALLOWED_VALUES restricts it to the cost centers you recognise, so that misspelled values cannot fragment your reports.

CREATE TAG cost_center
  ALLOWED_VALUES 'finance', 'marketing', 'engineering', 'product';

Next, you apply the tag. Tag the warehouses that a team owns outright. When a warehouse is shared, tag the users instead, because in that case query costs are attributed to users and the user's tag identifies the department. The documentation recommends automating tagging when resources and users are created, for example by provisioning users through a SCIM identity provider that sets the tag.

ALTER WAREHOUSE warehouse1 SET TAG cost_management.tags.cost_center='SALES';
ALTER WAREHOUSE warehouse2 SET TAG cost_management.tags.cost_center='SALES';
ALTER WAREHOUSE warehouse3 SET TAG cost_management.tags.cost_center='FINANCE';

ALTER USER finance_user SET TAG cost_management.tags.cost_center='FINANCE';
ALTER USER sales_user SET TAG cost_management.tags.cost_center='SALES';

To report on dedicated warehouses within one account, join ACCOUNT_USAGE.TAG_REFERENCES to WAREHOUSE_METERING_HISTORY on object_id and warehouse_id. The COALESCE to 'untagged' matters: it shows the spend that nobody has claimed yet.

SELECT
    TAG_REFERENCES.tag_name,
    COALESCE(TAG_REFERENCES.tag_value, 'untagged') AS tag_value,
    SUM(WAREHOUSE_METERING_HISTORY.credits_used_compute) AS total_credits
  FROM
    SNOWFLAKE.ACCOUNT_USAGE.WAREHOUSE_METERING_HISTORY
      LEFT JOIN SNOWFLAKE.ACCOUNT_USAGE.TAG_REFERENCES
        ON WAREHOUSE_METERING_HISTORY.warehouse_id = TAG_REFERENCES.object_id
          AND TAG_REFERENCES.domain = 'WAREHOUSE'

With several accounts, the tags must mean the same thing everywhere. You create them once in a key account, such as the organization account, and replicate the tag database to the other accounts through a replication group. Organization-wide reporting then runs from the organization account, because that is the only place the ORGANIZATION_USAGE TAG_REFERENCES view exists.

Sources4

3.Query tags and per-query cost for shared ML applications

Object tags break down when one application serves many teams. In that case, the application sets a query tag on its session, and every later query in that session carries the tag.

ALTER SESSION SET QUERY_TAG = 'COST_CENTER=finance';

The per-query figures come from ACCOUNT_USAGE.QUERY_ATTRIBUTION_HISTORY, where the cost of each query is the warehouse credit usage for executing it. You can group by query tag for applications, or join on user_name to TAG_REFERENCES for shared warehouses. A useful hygiene query finds credits from users with no tag. One limit is easy to miss: this view exists only per account, so per-query attribution cannot be run organization-wide from the organization account.

Sources4

4.What a feature view costs, and the settings that change it

To attribute feature cost at a fine grain, you first need to know what a feature view runs on. Snowflake-managed feature views are backed by dynamic tables, so their refreshes consume compute and their results consume storage, both on dynamic-table terms. External feature views are plain views, so they add no storage cost. If you enable online serving with hybrid tables, key lookups and ingestion consume virtual warehouse credits at standard rates. Cloud services are also used to detect changes in base objects and decide when a refresh is needed.

Each managed feature view is configured with a refresh_freq and a warehouse, and you can change both after registration. This gives you two levers. First, refreshing less often lowers cost when consumers can tolerate older data. Second, pointing the feature view at its own tagged warehouse makes its refresh credits visible as a separate line in the warehouse-tag report from the previous section.

fs.update_feature_view(
    name="<name>",
    version="<version>",
     ? ="<new_fresh_freq>",    # optional
    warehouse="<new_warehouse>",        # optional
    desc="<new_description>",           # optional
)

Sources56

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.Tagging the service user of a shared ML application is enough to split its cost across the departments it serves.Why is that wrong?

   A user tag names a single department. When one application queries on behalf of several departments, each query needs a query tag that identifies the department.

   Covered in Query tags and per-query cost for shared ML applications

2. 2.Organization-wide tag reports can be run from any account in the organization.Why is that wrong?

   The ORGANIZATION_USAGE TAG_REFERENCES view exists only in the organization account.

   Covered in Object tags on dedicated ML resources