What you will be able to do
- Configure automatic retries and auto-suspend on a task graph, and recover a failed run from the task that failed
- Pick between TASK_HISTORY, event-table task alerts and notification integrations to find out about failures
- Use ACCESS_HISTORY to audit which base tables and columns a query read or wrote
- Trace a bad value upstream to its source and downstream to everything it affects with Snowsight lineage
1.Retries, auto-suspend and recovering a failed graph
Scheduled pipelines in Snowflake are tasks: SQL or stored-procedure steps chained into a task graph under a root task that holds the schedule. By default, if any child task fails, the whole graph run counts as failed, and nothing retries it until the next scheduled run. On an hourly pipeline, one network error can therefore leave the data an hour stale. Two parameters on the root task change that.
TASK_AUTO_RETRY_ATTEMPTS retries the entire graph straight away, up to the number of times you set, when a child fails. Only when those attempts are used up does the run count as failed. SUSPEND_TASK_AFTER_NUM_FAILURES sets how many failed runs in a row it takes before Snowflake suspends the graph. The default is 10. Suspension is a circuit breaker. A source that is broken for good, such as a dropped table or revoked access, stops using compute after a handful of failures instead of failing every minute until someone notices.
CREATE OR REPLACE TASK task_root
SCHEDULE = '1 MINUTE'
TASK_AUTO_RETRY_ATTEMPTS = 2 -- Failed task graph retries up to 2 times
SUSPEND_TASK_AFTER_NUM_FAILURES = 3 -- Task graph suspends after 3 consecutive failures
AS SELECT 1;Once you have fixed the cause, you don't need to rerun the steps that already succeeded. EXECUTE TASK … RETRY LAST restarts the latest graph run from the task that failed, and its children then continue as normal. For an older run, use EXECUTE TASK … RETRY GRAPH RUN GROUP with that run's GRAPH_RUN_GROUP_ID. A step can also be bypassed on purpose: suspend that child task, and the graph carries on as if it had succeeded. This is how you keep the rest of a pipeline running while one broken step waits for a fix.
Checkpoint 1 of 7· Fill the gap
This hourly graph sometimes fails on a short network error. You want a failed graph run retried straight away instead of waiting for the next scheduled run. Which parameter goes in the blank on the root task?
CREATE OR REPLACE TASK task_root
SCHEDULE = '1 MINUTE'
? = 2 -- Failed task graph retries up to 2 times
SUSPEND_TASK_AFTER_NUM_FAILURES = 3 -- Task graph suspends after 3 consecutive failures
AS SELECT 1;TASK_AUTO_RETRY_ATTEMPTS on the root task makes Snowflake retry the whole graph straight away, up to the number you set. The run counts as failed only after those retries are used up. SUSPEND_TASK_AFTER_NUM_FAILURES controls when the graph is suspended, not whether it retries.
Source: docs.snowflake.comCheckpoint 2 of 7· Check yourself
One child task in a scheduled task graph keeps failing because its source table was dropped. The fix will take several days, and the other steps should keep loading in the meantime. What should you do?
A suspended child is treated as though it succeeded, so the rest of the graph carries on. Suspending the root stops the whole graph. Retries cannot fix a source that is broken for good. The failure threshold suspends the entire graph, not only the failing child.
“When you suspend a child task, the task graph continues to run as though the child task had succeeded.”Source: docs.snowflake.com
Sources1
2.Logging and monitoring task runs
TASK_HISTORY is where you look first to diagnose a failure. This Information Schema table function returns runs from the past seven days and scheduled runs up to eight days ahead, across the whole account, for one task (TASK_NAME) or for a whole graph (ROOT_TASK_ID). ERROR_ONLY => TRUE narrows it to failed runs. To find out who a task ran as, check the OWNER and EXECUTE_AS_USER columns of SHOW TASKS, then QUERY_HISTORY, which shows SYSTEM when a task did not run as a real user.
Pushing alerts out to people depends on the event table, and the event table records nothing until you configure it. LOG_EVENT_LEVEL controls which task events get recorded, and its default is OFF. A task alert computes an error rate as failed runs divided by recorded runs. At OFF nothing is recorded, so the alert never fires. At WARN or ERROR only failures are recorded, so one failure in 100 runs reads as a 100% error rate. Set INFO (or DEBUG/TRACE) on the task, its schema, its database or the account.
| Mechanism | Delivers to | Fires on | Use it for |
|---|---|---|---|
| Task alert | Email or webhook | Error rate across tasks crossing a threshold you choose | Watching overall task health |
| Notification integration (ERROR_INTEGRATION / SUCCESS_INTEGRATION) | Amazon SNS, Microsoft Azure Event Grid or Google Pub/Sub, on the account's own cloud | Each individual task failure and each successful graph run | Feeding task events into downstream systems |
You can use both mechanisms at once. Email and webhook integrations do not work as ERROR_INTEGRATION, so a team that wants failure emails needs a task alert. Notification delivery is at-least-once, so a system consuming these messages must cope with duplicates. NOTIFICATION_HISTORY lets you check what was sent. A finalizer task gives you a third, in-graph option: it runs after every other task has completed or failed, and it can send its own summary.
Checkpoint 3 of 7· Check yourself
A team creates a task alert with a 5% error-rate threshold. They don't change any logging parameters. Over the next day, several runs of the monitored task fail. What happens?
LOG_EVENT_LEVEL defaults to OFF. At OFF, no task events reach the event table, so the alert has nothing to evaluate. Setting it to WARN or ERROR would overstate the rate, because only failures would be recorded. INFO or lower records both successes and failures.
“At OFF, which is the default, no task events are recorded, so the alert never fires.”Source: docs.snowflake.com
Checkpoint 4 of 7· Check yourself
The on-call team wants an email whenever a task fails. An engineer suggests creating an email notification integration and setting it as the task's ERROR_INTEGRATION. What is the right approach?
ERROR_INTEGRATION and SUCCESS_INTEGRATION only push to Amazon SNS, Azure Event Grid or Google Pub/Sub. For email or webhook delivery, Snowflake directs you to a task alert. NOTIFICATION_HISTORY only lets you query what was already sent.
“To receive task notifications by email or webhook, use a task alert instead.”Source: docs.snowflake.com
3.Auditing reads and writes with ACCESS_HISTORY
Auditing means answering "who touched this data, and which columns?" ACCESS_HISTORY, in the ACCOUNT_USAGE and ORGANIZATION_USAGE schemas, has one row per SQL statement. Each row links the user, the query, the objects, the columns and any write. It captures reads, plus writes such as INSERT, UPDATE, DELETE and variants of COPY. Several of its columns are JSON arrays, and the question you are asking decides which one to read.
| Column | What it records |
|---|---|
| user_name | The user who issued the query |
| direct_objects_accessed | Tables, views, columns, UDFs and procedures directly named in the query, including through * |
| base_objects_accessed | All base data objects needed to execute the query, such as the tables behind a view |
| objects_modified | Objects associated with a write operation in the query |
When an auditor asks which physical tables sat behind a view, the answer comes from flattening base_objects_accessed for the queries that named the view. direct_objects_accessed would only show the view itself. Searching QUERY_TEXT in query history for the view name finds the queries but not what they read underneath, because the text names only the view. An object dependency, meaning a view referencing a table, is a structural relationship. It does not record which user read which table at query time.
Checkpoint 5 of 7· Check yourself
An auditor asks for the users who read the physical tables behind the view V_SALES_REPORT last quarter. Which approach answers this?
In ACCESS_HISTORY, base_objects_accessed lists the base tables a query needed in order to run, and each row also records the user. direct_objects_accessed shows only the view itself. Searching QUERY_TEXT finds the queries but not the tables they read underneath. An object dependency only records that the view references a table. It does not record who read which table at query time.
“Object dependencies, when an object references a base object but does not materialize or copy data, such as when a view references a table.”Source: docs.snowflake.com
Checkpoint 6 of 7· Exam question
An auditor asks which physical tables were read when analysts queried the view V_SALES_REPORT, which joins three base tables. The account is on Enterprise Edition. Which approach returns the underlying tables rather than only the view that was named in each query?
Correct answer: D — Read the BASE_OBJECTS_ACCESSED column of SNOWFLAKE.ACCOUNT_USAGE.ACCESS_HISTORY and flatten it to list the underlying tables for each query.
- A. Incorrect.
DIRECT_OBJECTS_ACCESSEDlists only the objects named directly in the query, so a query on the view shows the view and not its base tables. - B. Incorrect.
OBJECT_DEPENDENCIESshows static definition-time references between objects; it holds no per-query or per-user access information. - C. Incorrect. Matching query text is fragile and misses queries made through other views, and the
TABLESview lists table metadata rather than who read what. - D. Correct.
BASE_OBJECTS_ACCESSEDrecords the underlying tables and columns touched after views are resolved, which is what read-lineage requires.
4.Tracing a bad value with data lineage
Data lineage answers a different question from auditing: not who read the data, but where the data came from and where it went. Snowflake tracks two kinds of relationship. Data movement happens when CTAS, INSERT or MERGE copies or materializes rows from one object into another. Object dependencies exist when an object references another without copying data, as a view does with a table. A source is *upstream* of its target, and the target is *downstream* of the source.
Suppose a revenue dashboard suddenly shows negative totals. In Snowsight, open Catalog » Explorer, select the reporting table, and open the Lineage tab. Use +/- to expand upstream and find the objects that fed it. Select an arrow between two objects to see the SQL statement that built the downstream object, or the stored procedure or task that ran it. For a nested procedure, the panel shows both the Direct procedure and the Root procedure. Column lineage narrows the search further: View Lineage on a column lists every upstream or downstream column that shares lineage with it, and its Distance value counts the hops.
Once the source is fixed, go back downstream. The same column view lists every object that inherited the bad value and will need a rebuild or a refresh. Two gaps limit what you will see. Procedures called anonymously show no procedure details. And procedure and task details are not backfilled, so older lineage edges lack them. You can also see only what your privileges allow: objects you cannot access are collapsed into a separate group.
A dashed border means the downstream column, which shares lineage with the tagged column, is missing the tag. Select Review and Apply to add it from the Lineage tab, so the sensitive column is protected wherever its data has flowed.
Checkpoint 7 of 7· Match them up
Match each data lineage term to an example of it in the pipeline that feeds a revenue dashboard.
Tap a term, then the definition that fits it.
Data movement means rows are copied or materialized, as with CTAS, INSERT or MERGE. An object dependency means one object references another without copying data, as a view does with a table. A source is upstream of its target, and the target is downstream of the source. In the column lineage view, Distance counts how many hops away a related column is.
“Object dependencies, when an object references a base object but does not materialize or copy data, such as when a view references a table.”Source: docs.snowflake.com
Sources7
Sources
(7)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“By default, if a child task fails, the entire task graph is considered to have failed.”
↩︎ Retries, auto-suspend and recovering a failed graph“the entire task graph is immediately retried, up to the number of times specified”
↩︎ Retries, auto-suspend and recovering a failed graph“By default, a task graph is suspended after 10 consecutive failures.”
↩︎ Retries, auto-suspend and recovering a failed graph“use EXECUTE TASK … RETRY LAST to attempt to run the task graph from the last failed task”
↩︎ Retries, auto-suspend and recovering a failed graph“When you suspend a child task, the task graph continues to run as though the child task had succeeded.”
↩︎ Exam trap 3“When you suspend a child task, the task graph continues to run as though the child task had succeeded.”
↩︎ Checkpoint - 2.
“This function can return all executions run in the past seven days or the next scheduled execution within the next eight days.”
↩︎ Logging and monitoring task runs - 3.
“To see who ran a task, use the QUERY_HISTORY view.”
↩︎ Logging and monitoring task runs - 4.
“At WARN or ERROR, only failed runs are recorded, so the alert overstates the error rate.”
↩︎ Logging and monitoring task runs“The email and webhook notification integration types aren’t supported for ERROR_INTEGRATION or SUCCESS_INTEGRATION.”
↩︎ Logging and monitoring task runs“Snowflake guarantees at-least-once message delivery of notifications.”
↩︎ Logging and monitoring task runs“At OFF, which is the default, no task events are recorded, so the alert never fires.”
↩︎ Exam trap 2“At OFF, which is the default, no task events are recorded, so the alert never fires.”
↩︎ Checkpoint“To receive task notifications by email or webhook, use a task alert instead.”
↩︎ Checkpoint - 5.
“The user access history can be found by querying the ACCESS_HISTORY view in the ACCOUNT_USAGE and ORGANIZATION_USAGE schemas.”
↩︎ Auditing reads and writes with ACCESS_HISTORY“such as the underlying tables that the data for the query comes from”
↩︎ Auditing reads and writes with ACCESS_HISTORY - 6.
“A JSON array that specifies the objects that were associated with a write operation in the query.”
↩︎ Auditing reads and writes with ACCESS_HISTORY“directly named in the query explicitly or through shortcuts”
↩︎ Exam trap 1“A JSON array of all base data objects to execute a query”
↩︎ Prediction - 7.
“Object dependencies, when an object references a base object but does not materialize or copy data, such as when a view references a table.”
↩︎ Auditing reads and writes with ACCESS_HISTORY“Data movement, such as when data is copied or materialized from one object to another.”
↩︎ Tracing a bad value with data lineage“A stored procedure or task can result in lineage between an upstream object and a downstream object.”
↩︎ Tracing a bad value with data lineage“For a given column, you can determine all upstream and downstream columns that share lineage with the column.”
↩︎ Tracing a bad value with data lineage“Details about stored procedures and tasks are not backfilled.”
↩︎ Tracing a bad value with data lineage“If a tag has a dashed border, the column does not have the tag applied.”
↩︎ Tracing a bad value with data lineage“Object dependencies, when an object references a base object but does not materialize or copy data, such as when a view references a table.”
↩︎ Checkpoint