What you will be able to do
- Trace upstream and downstream lineage for tables and columns in Snowsight
- Query lineage programmatically with SNOWFLAKE.CORE.GET_LINEAGE
- Choose between Elastic and Named Channels for Snowpipe Streaming feature ingestion
- Read the Feature Store Offline Refresh and Stream Ingest tabs to diagnose stale or failing feature pipelines
Key concept
Upstream and downstream lineage — Every lineage relationship has a direction. The object data came from is upstream, and the object it flows into is downstream. Impact analysis, root-cause tracing and freshness checks all come down to following that direction one hop at a time.
1.Visualizing lineage in Snowsight
A feature table almost never stands alone. It is built from raw ingestion tables, joined to reference data and read by views and training sets. When a feature looks wrong, the first question is where its data came from. Snowflake answers that with data lineage, which records how data flows from source objects to target objects.
Lineage captures two kinds of relationship, and the difference matters. Data movement happens when data is copied or materialized from one object into another, for example with CTAS, INSERT or MERGE. An object dependency exists when one object references another without copying its data, as a view does with its base table.
To open the graph, go to Catalog » Explorer, pick a table or view, and select the Lineage tab. Snowsight reveals the graph one step at a time. Use +/- to expand objects further upstream or downstream. Select an arrow to see how the downstream object was created, such as the SQL statement, subject to your privileges. Nodes are grouped by database and then schema, external objects are grouped by vendor, and objects you cannot access go into a separate group that is collapsed by default.
Column lineage takes this down to individual columns. Select an object, hover over a column in the side panel, choose View Lineage, then pick Upstream Lineage or Downstream Lineage. The Distance column shows how many hops away each related column is. A downstream distance of 1 means the object was created directly from the current one. This view also flags missing or mismatched tags on related columns and offers to apply them.
If a stored procedure or task created the downstream object, selecting the arrow shows a Direct procedure and, for nested calls, a Root procedure. There are two gaps. Anonymously called procedures do not appear, and procedure and task details are not backfilled for lineage that existed before Snowflake added that support.
Checkpoint 1 of 7· Check yourself
Which operation creates a data movement lineage relationship rather than an object dependency?
MERGE, INSERT and CTAS copy or materialize data into the target, so they count as data movement. A view only references its base table.
“CREATE TABLE AS SELECT (CTAS), INSERT, or MERGE operations on tables result in data movement.”Source: docs.snowflake.com
Checkpoint 2 of 7· Put it in order
Put the steps for tracing a single column's downstream lineage in order
- 1.Open the Lineage tab and select the object that contains the column
- 2.Read the Distance column to see how far away each related column is
- 3.Hover over the column name in the side panel and select View Lineage
- 4.Select Downstream Lineage
The side panel opens when you select the object. View Lineage is reached from the column name there, and the direction choice lists the related columns along with their distance.
“Hover over the column name in the side panel, and select View Lineage.”Source: docs.snowflake.com
Sources1
2.Querying lineage with GET_LINEAGE
The Snowsight graph is good for exploring by hand. For automation, such as checking every downstream feature table before you change a source, query lineage with the SNOWFLAKE.CORE.GET_LINEAGE table function. You pass a starting object, its domain, a direction and a maximum distance. You get back one row per edge in the lineage graph.
SELECT
DISTANCE,
SOURCE_OBJECT_NAME,
TARGET_OBJECT_NAME
FROM TABLE (SNOWFLAKE.CORE.GET_LINEAGE(
object_name => 'my_database.sch.table_a',
object_domain => 'TABLE',
direction => 'DOWNSTREAM',
max_distance => 2));Three behaviours are worth knowing. First, an object with no recorded lineage returns an empty result rather than an error. You do get an error if the object does not exist, is not accessible, does not support lineage, or is not in the domain you named. Second, output is capped at 10 million edges and anything beyond that is cut off silently, so very wide graphs need a smaller max_distance. Third, lineage ingested from external systems appears alongside native lineage. The origin key in the object details tells you which is which.
Checkpoint 3 of 7· Fill the gap
Complete the call so it returns the objects built from table_a, not the ones that feed it.
SELECT
DISTANCE,
SOURCE_OBJECT_NAME,
TARGET_OBJECT_NAME
FROM TABLE (SNOWFLAKE.CORE.GET_LINEAGE(
object_name => 'my_database.sch.table_a',
object_domain => 'TABLE',
direction => ' ? ',
max_distance => 2));Objects created from table_a are its targets, which are downstream of it. UPSTREAM would return the sources that feed table_a.
Source: docs.snowflake.comCheckpoint 4 of 7· Check yourself
GET_LINEAGE on an accessible table that supports lineage returns zero rows. What does that mean?
An empty result is a normal outcome. Missing privileges or a wrong domain raise an error, and oversized output is truncated, not dropped.
“The output table contains no rows if no lineage information is available for the specified object; this is not an error.”Source: docs.snowflake.com
Sources2
3.Streaming ingestion for near real-time features
Fresh features need fresh data. Snowpipe Streaming is Snowflake's real-time ingestion service. Applications stream rows directly into Snowflake tables or Snowflake-managed Iceberg tables. Because rows arrive directly, the pipeline can skip staging files, intermediate object storage, message buses and connector services the workload doesn't otherwise need. The documented ceilings are up to 20 GB/s per table and ingest-to-queryable latency as low as 5 seconds, though real results depend on row size, table width, batching, concurrency and transformations.
In both of its modes, a channel is the logical path that carries rows through a pipe to a target table. The design choice is which kind of channel to use.
| Mode | Who manages channels | Delivery guarantee | Choose it when |
|---|---|---|---|
| Elastic Channels | Snowflake manages them and scales with traffic | At-least-once, no ordering guarantee | Most new applications; the recommended starting point |
| Named Channels | The application, using offset tokens | Ordered, exactly-once within each channel | The source requires strict ordering, e.g. Kafka partitions or CDC |
Checkpoint 5 of 7· Check yourself
A feature pipeline consumes change-data-capture events whose order must be preserved, and duplicates would corrupt aggregates. Which Snowpipe Streaming mode fits?
Elastic Channels are at-least-once with no ordering guarantee. CDC needs the ordering and exactly-once semantics of Named Channels.
“Named Channels provide ordered, exactly-once ingestion within each channel by using offset tokens.”Source: docs.snowflake.com
Checkpoint 6 of 7· Exam question
A data engineer plans to rename the `amount_usd` column in a raw orders table that feeds several feature views, datasets, and registered models. Which TWO approaches identify the downstream ML objects that the change would affect?(Select 2)
Correct answers: A, E — Open the table in Snowsight, switch to the Lineage tab, and trace column lineage for `amount_usd` downstream to feature views, datasets, and models.; Call `SNOWFLAKE.CORE.GET_LINEAGE` with the table's qualified name, its object domain, and the `DOWNSTREAM` direction to list dependent objects.
- A. Correct: Snowsight lineage covers tables, feature views, datasets, and models, and column lineage shows which downstream columns derive from the renamed one.
- B. Incorrect: that view records definitional dependencies such as a view referencing a table, and it does not track datasets or models built from data copied out of the table.
- C. Incorrect: grants say who may read the table, not which objects were derived from it, so roles do not map to downstream features or models.
- D. Incorrect: text search over a short window misses consumers that ran outside it and cannot link queries to registered models or datasets.
- E. Correct: the GET_LINEAGE table function exposes the same lineage graph programmatically, so downstream feature views, datasets, and models can be listed in a query.
Sources3
4.Monitoring feature pipelines in the Feature Store view
Once data is flowing, you need to know whether features are actually being refreshed and served. In Snowsight, go to AI & ML » Features and select a feature store. Each tab runs its monitoring queries on a warehouse you choose. There are three tabs.
Offline Refresh lists the refresh jobs of the dynamic tables behind each feature view. The Source Data Timestamp column shows the timestamp of the upstream data that triggered each refresh, which makes it your freshness signal. Each row also shows status, duration, inserted and deleted row counts, and a link to the dynamic table's refresh history. Refreshes with no changed rows are hidden unless you select *Include empty refreshes*. A Failed status needs attention: after 5 consecutive scheduled failures, Snowflake suspends the dynamic table, and no refreshes run until someone resumes it. The tab reads INFORMATION_SCHEMA.DYNAMIC_TABLE_REFRESH_HISTORY, which keeps 7 days of history. For up to 365 days, query the DYNAMIC_TABLE_REFRESH_HISTORY view directly.
Online Serving shows request volume, error percentage and p50/p90/p99 latency for online retrieval. It appears only when online serving is enabled for at least one feature view.
Stream Ingest appears only when at least one stream feature view exists, and both online tabs require the online service to be running. It charts ingest responses per second, responses by HTTP code, p50 to p99 latency, and concurrency, meaning ingest QPS divided by the number of provisioned nodes (3 by default, autoscaling on CPU).
| Symptom | Likely cause |
|---|---|
| High p99 latency | Large payload per request, or a burst of concurrent requests exceeding available capacity |
| Elevated 5xx errors | Runtime errors or resource throttling; check the Concurrency chart |
| Elevated 4xx errors | Malformed request payload or a schema mismatch between the payload and the feature view schema |
| Flat or zero ingest rate | Upstream source stopped producing events, or the ingest endpoint is unreachable |
Checkpoint 7 of 7· Match them up
Match each Stream Ingest observation to its most likely cause
Tap a term, then the definition that fits it.
429 is the throttling code. Other 4xx errors point to the request payload, 5xx errors point to the service side, and a flat rate means no events are arriving.
“A spike in 429 responses indicates that the ingest rate is exceeding the provisioned limit.”Source: docs.snowflake.com
Sources4
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.Every stored procedure that produces a table shows up in the lineage arrow details.Why is that wrong?
Procedures called anonymously are not shown, and procedure and task details are not backfilled for lineage that predates the feature.
Covered in Visualizing lineage in Snowsight
2.An Elastic Channel acknowledgement means the row is already queryable.Why is that wrong?
The acknowledgement only confirms durable buffering. Table processing and query visibility happen afterwards.
3.A feature view whose refreshes keep failing will keep retrying until the source is fixed.Why is that wrong?
After five consecutive scheduled failures, the backing dynamic table is suspended and must be resumed manually.
Covered in Monitoring feature pipelines in the Feature Store view
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“Details about stored procedures and tasks are not backfilled.”
↩︎ Visualizing lineage in Snowsight“Objects you don’t have access to are placed in a separate group that is collapsed by default.”
↩︎ Visualizing lineage in Snowsight“In lineage terminology, the source object is “upstream” of the target object, and the target object is “downstream” of the source object.”
↩︎ Key concept“If you call a stored procedure anonymously, details about the stored procedure do not appear in the lineage.”
↩︎ Exam trap 1“Object dependencies, when an object references a base object but does not materialize or copy data, such as when a view references a table.”
↩︎ Prediction“CREATE TABLE AS SELECT (CTAS), INSERT, or MERGE operations on tables result in data movement.”
↩︎ Checkpoint“Hover over the column name in the side panel, and select View Lineage.”
↩︎ Checkpoint - 2.
“If there are more than 10 million rows in the output, the function silently truncates output to 10 million rows.”
↩︎ Querying lineage with GET_LINEAGE“NATIVE for a Snowflake object, or OPEN_LINEAGE for an object whose lineage was ingested from an external source.”
↩︎ Querying lineage with GET_LINEAGE“The output table contains no rows if no lineage information is available for the specified object; this is not an error.”
↩︎ Checkpoint - 3.https://docs.snowflake.com/en/user-guide/snowpipe-streaming/data-load-snowpipe-streaming-overviewOfficial docs
“Elastic Channels provide at-least-once delivery without an ordering guarantee.”
↩︎ Streaming ingestion for near real-time features“As low as 5 seconds ingest-to-queryable latency”
↩︎ Streaming ingestion for near real-time features“An acknowledgement confirms that Snowflake has durably buffered the append, so the producer can release its retained copy; table processing and query visibility follow.”
↩︎ Exam trap 2“Named Channels provide ordered, exactly-once ingestion within each channel by using offset tokens.”
↩︎ Checkpoint - 4.
“The Offline Refresh tab uses information from INFORMATION_SCHEMA.DYNAMIC_TABLE_REFRESH_HISTORY, which retains data for 7 days.”
↩︎ Monitoring feature pipelines in the Feature Store view“Malformed request payload or a schema mismatch between the payload and the feature view schema.”
↩︎ Monitoring feature pipelines in the Feature Store view“After 5 consecutive scheduled failures, Snowflake automatically suspends the dynamic table.”
↩︎ Exam trap 3“A spike in 429 responses indicates that the ingest rate is exceeding the provisioned limit.”
↩︎ Checkpoint