What you will be able to do
- Restructure notebook code into modular, parameterized pipeline steps with a local entrypoint script
- Run each pipeline step as an ML Job whose result and runtime are fixed so the steps can be chained
- Build a task graph in which a failed validation step stops training and deployment
- Pass values between tasks, trigger a graph on new data, and choose an overlap policy and finalizer behaviour
- Resume, run, retry and monitor a task graph, and start runs by hand from Snowflake CLI, SnowSQL and the Python SDK
Key concept
Task graph (DAG) — A root task that decides when the pipeline runs, plus child tasks linked to their parents with AFTER. A child starts only after every parent has succeeded, so the order of the graph is also its quality gate.
1.From experiment to an automated ML pipeline
An ML workflow has four stages: data preparation, feature engineering, model development and deployment. Each one starts life as an interactive experiment. Once a model proves its value, the work moves to operationalization, where the pipeline is hardened and automated so that every change to code, data or model is built, tested and deployed the same way each time. In Snowflake, the orchestration layer for this is a task graph: the pipeline turned into a directed acyclic graph that runs on a schedule or when an event triggers it.
A task graph has one root task and any number of dependent child tasks. The root task decides when the graph runs. Each child names its parents with CREATE TASK ... AFTER, and dependencies always run from start to finish, with no loops. Children that share a parent run in parallel. A child with several parents waits until all of them have finished successfully. One graph can hold up to 1000 tasks, and a single task can have at most 100 parents and 100 children.
CREATE TASK task_root
SCHEDULE = '1 MINUTE'
AS SELECT 1;
CREATE TASK task_a
AFTER task_root
AS SELECT 1;
CREATE TASK task_b
AFTER task_root
AS SELECT 1;
CREATE TASK task_c
AFTER task_a, task_b
AS SELECT 1;Every task in a graph must have the same owner and live in the same database and schema. Ownership matters more than it looks. Granting ownership of one task to a different role cuts that task's links to its parents and children. Moving the whole graph to a new owner keeps the links intact, either by GRANT OWNERSHIP on all tasks in the schema or by dropping the owning role. If parallel children share a user-managed warehouse, size that warehouse for the concurrent runs.
Checkpoint 1 of 11· Fill the gap
This task graph needs task_a and task_b to run in parallel straight after the root, and task_c to wait for both of them. Which task name fills the blank?
CREATE TASK task_root
SCHEDULE = '1 MINUTE'
AS SELECT 1;
CREATE TASK task_a
AFTER task_root
AS SELECT 1;
CREATE TASK task_b
AFTER ?
AS SELECT 1;
CREATE TASK task_c
AFTER task_a, task_b
AS SELECT 1;Children that share a parent run in parallel, so task_b must name task_root as its parent, just as task_a does. Naming task_a would make the two tasks run one after the other. Naming task_c would create a loop, and task graphs cannot have loops. A task cannot be its own parent either.
Source: docs.snowflake.com2.Restructuring notebook code into modular steps
A notebook cannot become a pipeline as it stands. Each major step needs to be its own function with clear inputs and outputs: data preparation, feature engineering, training and evaluation. Hard-coded values become parameters. Table names, stages and hyperparameters come from configuration, so the same code runs in dev and in prod. Keeping steps modular from the first interactive session, with one cell or function per stage, makes this refactor small. Deciding where that code should live, whether in Workspaces, stored procedures or Python files, is covered in When to develop in Workspaces vs stored procedures, SQL or Python files. For a pipeline, the cells end up as modules under a package such as src/ml_pipeline/, with data/, features/, models/ and inference/ packages.
Write an entrypoint script as well, one that runs the whole pipeline locally. It is the version you debug, and the graph definition (dag.py) sits next to it in scripts/. The entrypoint reads --config and --env, loads the matching configuration, and calls the steps in order. Validation sits between loading and feature creation, so bad data never reaches training.
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--config", required=True, help="Config file path")
parser.add_argument("--env", default="dev", help="Environment (dev/prod)")
args = parser.parse_args()
# Load configuration
config = load_config(args.config, args.env)
# Execute pipeline stages
raw_data = load_raw_data(config.data.source_table)
validate_data_quality(raw_data, config.data.quality_checks)
features = create_features(raw_data, config.features.transformations)
model = train_model(features, config.model.hyperparameters)
metrics = evaluate_model(model, features, config.model.eval_metrics)
register_model(model, metrics, config.model.registry_name)Checkpoint 2 of 11· Check yourself
A data scientist's training notebook has the dev source table name and the XGBoost hyperparameters typed straight into its cells. The team wants the same code to run as a pipeline in both dev and prod. What should they change before building the task graph?
Production-ready code splits each major step into its own function and parameterizes every configuration value, including table names as well as hyperparameters. Then one codebase can load the dev or prod config, for example through --env, without copies of the code or hard-coded values.
“Parameterize all configuration values like table names and hyperparameters to enable cross-environment deployment.”Source: docs.snowflake.com
Sources1
3.Running each step as an ML Job
Heavy steps such as training are dispatched to Snowflake compute as ML Jobs, through a @remote function or submit_file / submit_directory / submit_from_stage. The mechanics are covered in Snowflake ML Jobs. Here the question is what a pipeline needs from a job for it to be chainable, and there are three things.
A result to collect. A @remote function returns its value directly. A file-based job sets the special __return__ variable instead. The pipeline then calls MLJob.result(), which blocks until the job reaches a terminal state and then either returns the value or raises an exception if the job failed. That exception matters for orchestration: a task that calls result() on a failed training job fails too, and the steps after it do not run.
A pinned runtime. If you leave out runtime_environment, the job runs on the latest Container Runtime available on the compute pool. That runtime can change between Tuesday's run and Wednesday's. Pinning a full version string such as "2.3.0", or a custom image with "cre@<name>", keeps unattended runs reproducible.
Imports in the right place. Most notebook code runs unchanged in an ML Job. The exception is the distributed ML APIs that exist only inside the Container Runtime: import them inside the job payload, not at the top of the orchestrating script, or the import fails.
Checkpoint 3 of 11· Check yourself
In a task graph, a train task submits training as a file-based ML Job, and an evaluate task runs AFTER it. One night the training job fails on the compute pool. What stops the evaluate task from running on a model that was never trained?
Submitting a job only returns an MLJob object. result() is the call that blocks until the job finishes and raises an exception if it failed, which makes the task fail, and a child starts only after its parents succeed. A returned value is just data and does not change which tasks run. Pinning the runtime makes runs reproducible but does not report failures.
“then returns the payload’s return value or, if execution failed, raises an exception.”Source: docs.snowflake.com
4.Gating training on a validation step
The typical ML graph chains VALIDATE_DATA → TRAIN_MODEL → DEPLOY_MODEL. The validation step can run Data Metric Functions and expectations against the new data. In the graph, its job is to act as a gate: when a check fails, training and deployment must not run.
Failure is the gate. A child task starts only once its predecessors have completed successfully, and by default one failed child marks the whole graph run as failed. So the validation procedure should raise an exception when a threshold is breached. Its task then fails, TRAIN_MODEL and DEPLOY_MODEL never start, and the failure is recorded in task history where anyone can see it.
The alternatives break in specific ways. Returning the text FAILED produces a successful task, so the child starts anyway. A return value only affects control flow if the child's own body contains logic that reads it and acts on it. A separate validation root task does not work either, because each root task schedules its own graph and nothing connects the two. A check in the finalizer comes too late, since the finalizer runs after training has already finished.
Use a deliberate branch when the outcome is a decision rather than an error. The Snowflake end-to-end example evaluates the trained model and picks the next task by name. If the model clears the quality threshold, the promotion task registers it and sets it as the default version, so callers of the model pick it up without changing their code (see the Model Registry). If it falls short, an alert task runs instead.
def check_model_quality(session: Session) -> str:
if metrics[config.metric_name] >= threshold:
return "promote_model" # High quality → promote
else:
return "send_alert" # Low quality → alertCheckpoint 4 of 11· Fill the gap
After evaluation, the pipeline should either promote the model or send an alert. The model clearing the bar is a decision, not an error. Which return type completes this branch task from the Python task API?
def check_model_quality(session: Session) -> ? :
if metrics[config.metric_name] >= threshold:
return "promote_model" # High quality → promote
else:
return "send_alert" # Low quality → alertA branch task returns the name of the next task to run as a string, such as "promote_model" or "send_alert". A True/False value or no return value would not name the task to run. Use a deliberate branch like this for a decision. A failed data check should still raise an error so that its children never start.
Source: www.snowflake.comCheckpoint 5 of 11· Exam question
A nightly task graph chains VALIDATE_DATA, TRAIN_MODEL, and DEPLOY_MODEL. When the validation step finds a null rate above the agreed threshold, training and deployment must not run, and the failure must show up in task history. How should the validation step be built?
Correct answer: C — Have the validation procedure raise an exception on a threshold breach, so its task fails and the dependent training and deployment tasks do not run.
- A. Incorrect. A SCHEDULE parameter has no knowledge of another root task's outcome, so training would run regardless.
- B. Incorrect. A return value never gates successors by itself; the child runs unless it explicitly reads the value and acts on it.
- C. Correct. A failed task prevents its successors from running in that run, and the error is recorded in task history.
- D. Incorrect. A finalizer runs after the other tasks finish, so training and deployment would already have happened.
5.Passing values between tasks and triggering on new data
Tasks hand data to each other through return values. A task calls SYSTEM$SET_RETURN_VALUE('<string>'), and any task that names it in its AFTER clause reads the value with SYSTEM$GET_PREDECESSOR_RETURN_VALUE. This is how TRAIN_MODEL passes the version name it just created to DEPLOY_MODEL in the same graph run. The value is a string of at most 10 kB, so pass an identifier such as a version name or a table name, not the model itself. In the Python task API the same handoff goes through the TaskContext.
The root task decides when the graph runs, either on a schedule or on an event. For retraining on newly arrived data, a triggered task is the better fit when the arrival time is unpredictable, for example when rows land continuously through Snowpipe. You create a stream on the source table, and the root task names that stream in a WHEN clause and has no SCHEDULE. A triggered task uses no compute until the stream has changes, so nothing is spent polling an empty table. Streams on tables, views, dynamic tables, Iceberg tables and data shares can trigger tasks. Hybrid tables and streams on external tables cannot.
| Root task setting | Runs when | Fits |
|---|---|---|
| SCHEDULE = '1 MINUTE' (or a longer interval) | The interval elapses, whether or not there is new data | Nightly or periodic retraining on a fixed cadence |
| WHEN on a stream, no SCHEDULE | The stream records a change | Retraining only when new training rows arrive |
Checkpoint 6 of 11· Check yourself
New labelled rows arrive in a training table through Snowpipe at unpredictable times, sometimes several times a day and sometimes not for a week. The team wants the retraining graph to run only when new rows have landed and to spend nothing in between. How should they set up the graph?
The root task decides when the graph runs. A root with a WHEN clause on a stream and no SCHEDULE runs only when the stream records changes, and it uses no compute while nothing changes. A one-minute schedule runs whether or not there is new data. A returned string does not stop a child from running. A second root task schedules its own separate graph and cannot start a task in this one.
“Task graphs can run on a recurring schedule, or they can be triggered by an event.”Source: docs.snowflake.com
6.Finalizer tasks and overlapping runs
A finalizer is an optional last task that runs after every other task in the graph run has finished, whether they succeeded or failed. That makes it the place for cleanup, such as dropping temporary training tables, and for sending a success or failure notification. You attach it to the root task with FINALIZE, not AFTER.
CREATE TASK task_finalizer
FINALIZE = task_root
AS SELECT 1;A finalizer has strict rules:
- Each root task has at most one finalizer, and a finalizer belongs to exactly one root.
- A finalizer cannot have child tasks.
- It is scheduled only once no other task in the run is running or queued.
- If the root task is skipped, for example because of an overlapping run, the finalizer is not started at all.
Because the finalizer runs on failure as well as success, it is the wrong place for model promotion, which should happen only when the run succeeds. Put promotion on the success path of the graph instead.
Overlap is the other run-level setting. By default only one instance of a graph runs at a time, and the root is scheduled again only after every task has finished. If a training run takes longer than the schedule interval, at least one scheduled run is skipped. The OVERLAP_POLICY parameter on the root task changes this behaviour.
| Value | Behaviour | When it is safe |
|---|---|---|
| NO_OVERLAP (default) | Runs serially. The next root run is scheduled only after all child tasks finish | Retraining that writes models or tables a second run would clash with |
| ALLOW_CHILD_OVERLAP | A new graph instance starts while children of the previous one are still running. Root tasks never overlap | Child steps can safely run alongside the previous run's children |
| ALLOW_ALL_OVERLAP | Several full instances, root included, run concurrently | Overlapping reads and writes cannot produce incorrect or duplicate data |
When overlap would be unsafe, the fix is to make sure each run finishes before the next one is due. Lengthen the schedule interval, size the warehouse up, or use serverless compute.
Checkpoint 7 of 11· Match them up
A nightly retraining graph sometimes runs past its next scheduled start time. Match each OVERLAP_POLICY value on the root task to what happens when that occurs.
Tap a term, then the definition that fits it.
With the default NO_OVERLAP, a missed run is skipped, not queued to catch up later. ALLOW_CHILD_OVERLAP lets only child tasks overlap with the previous run. ALLOW_ALL_OVERLAP lets whole graph instances run in parallel, so it is safe only when overlapping reads and writes cannot produce incorrect or duplicate data.
“OVERLAP_POLICY = ALLOW_ALL_OVERLAP: Allows unlimited true parallelism. Snowflake can run multiple instances of the entire task graph, including the root task, concurrently.”Source: docs.snowflake.com
Checkpoint 8 of 11· Exam question
Which statements about a finalizer task in a Snowflake task graph are true? (Select two.)(Select 2)
Correct answers: A, E — Each root task can have only one finalizer task, and that finalizer task cannot itself have any child tasks below it in the graph.; A finalizer runs after all other tasks in the graph run have finished, succeeded or failed, so it suits cleanup such as dropping temporary tables.
- A. Correct. Both restrictions are part of the finalizer definition for a task graph.
- B. Incorrect. If the root task is skipped, the finalizer does not start for that run.
- C. Incorrect. A finalizer also runs after failures, so promotion placed there could publish a model from a broken run.
- D. Incorrect. A finalizer is attached to the root task and runs as part of each graph run, not on a separate schedule.
- E. Correct. This is the purpose of a finalizer: guaranteed post-run work regardless of the outcome of the other tasks.
Sources2
7.Starting, running and retrying a task graph
New tasks are created suspended, and resuming the root does not resume its children. There are two ways to enable a graph. You can run ALTER TASK ... RESUME on each child, including the finalizer, and then on the root. Or you can make one call: SELECT SYSTEM$TASK_DEPENDENTS_ENABLE('<root_task_name>'), which resumes the root and every dependent task. To list a graph's tasks, pass the root task to the TASK_DEPENDENTS table function.
EXECUTE TASK on the root starts one run immediately, which is useful for testing a changed graph before it goes on a schedule. It does not change any task's state. Only children that are already resumed take part in the run. To change a task in a scheduled graph, suspend the root first. A run already in progress finishes, and future runs are cancelled until the root is resumed. Suspending a single child skips it: the graph continues as if that child had succeeded.
Failures can be retried without starting over:
EXECUTE TASK ... RETRY LASTreruns the latest graph run from the task that failed, so a fixed TRAIN_MODEL runs again without repeating a VALIDATE_DATA that already passed.EXECUTE TASK ... RETRY GRAPH RUN GROUPwith aGRAPH_RUN_GROUP_IDretries an older run.TASK_AUTO_RETRY_ATTEMPTSon the root retries the whole graph immediately when a child fails.SUSPEND_TASK_AFTER_NUM_FAILURESsets how many consecutive failures suspend the graph. The default is 10.
CREATE OR REPLACE TASK task_root
SCHEDULE = '1 MINUTE'
TASK_AUTO_RETRY_ATTEMPTS = 2 -- Failed task graph retries up to 2 times
SUSPEND_TASK_AFTER_NUM_FAILURES = 3 -- Task graph suspends after 3 consecutive failures
AS SELECT 1;To find out which step failed, open Snowsight under Monitoring > Task History and select the graph. The view shows each task's status, logs and dependencies for the latest run, and which branch a conditional step took.
Checkpoint 9 of 11· Fill the gap
When a child task such as TRAIN_MODEL fails during the night, the team wants the whole graph to retry straight away instead of waiting for the next scheduled run. They also want the graph to suspend after three failures in a row. Which root-task parameter fills the blank?
CREATE OR REPLACE TASK task_root
SCHEDULE = '1 MINUTE'
? = 2 -- Failed task graph retries up to 2 times
SUSPEND_TASK_AFTER_NUM_FAILURES = 3 -- Task graph suspends after 3 consecutive failures
AS SELECT 1;Setting TASK_AUTO_RETRY_ATTEMPTS on the root task makes the whole graph retry immediately when a child fails, up to the number given. SUSPEND_TASK_AFTER_NUM_FAILURES does a different job: it only decides how many consecutive failures suspend the graph. OVERLAP_POLICY controls whether runs can overlap and has nothing to do with retries.
Source: docs.snowflake.comCheckpoint 10 of 11· Exam question
A team defined a graph of six tasks with CREATE TASK ... AFTER statements. They want every task in the graph enabled with one call after deployment. Which approach does that?
Correct answer: B — Run SELECT SYSTEM$TASK_DEPENDENTS_ENABLE on the root task's name, which resumes the root along with all of its dependent tasks.
- A. Incorrect. Resuming the root does not resume children; suspended children are skipped when the graph runs.
- B. Correct. This system function enables the root and every dependent task in one call.
- C. Incorrect. The OPERATE privilege controls who may resume tasks; it never resumes any task by itself.
- D. Incorrect. EXECUTE TASK triggers a run; it does not change the suspended or started state of child tasks.
Sources2
8.Driving pipelines from Snowflake CLI, SnowSQL and the SDKs
While you develop a pipeline, you start runs by hand from whatever client is nearest. ML Jobs can be submitted from any development environment, such as VS Code or Jupyter. They need a Snowpark Session, which is built from the connection in ~/.snowflake/config.toml. Tasks and task graphs can be defined in SQL, JavaScript, Python, Java, Scala or Snowflake Scripting. Wiring these commands into automated deployment is covered in Snowflake CLI for automation.
| Tool | Status | Typical use in an ML pipeline |
|---|---|---|
| Snowflake CLI | Preferred client for new installs and new work | Run SQL such as EXECUTE TASK or SYSTEM$TASK_DEPENDENTS_ENABLE, and manage stored procedures and stages |
| SnowSQL | Legacy client, supported in version 1.5.x through April 16, 2028. New features go only to Snowflake CLI | Run the same SQL statements in existing scripts |
| Python SDK (snowflake-ml-python, Snowpark) | Required for ML Jobs, version 1.26.0 and later | Submit ML Jobs and define the graph with the DAG context manager, >> dependencies, TaskContext and DAGTaskBranch |
The Python route works well for iterating. In Snowflake's end-to-end example, pipeline_local.py runs the pipeline without any orchestration, and the training ML Job shows up in the Snowsight Job UI. pipeline_dag.py then deploys the same steps as a task graph. Its --run-dag flag deploys the graph, runs it once, follows its progress and prints the final status. Its --schedule flag deploys the graph to run on a recurring interval.
python src/pipeline_dag.py --schedule 1d # Daily execution
python src/pipeline_dag.py --schedule 12h # Every 12 hours
python src/pipeline_dag.py --schedule 30m # Every 30 minutesCheckpoint 11 of 11· Match them up
Your team is automating a retraining pipeline and has to choose a tool for each job. Match each tool to the role it should play.
Tap a term, then the definition that fits it.
Snowflake CLI is the preferred command-line client, and new features go only to it. SnowSQL is a legacy client that is still supported for existing scripts. Submitting ML Jobs and defining graphs in Python require snowflake-ml-python 1.26.0 or later. In the example, the --run-dag flag deploys the graph and runs it a single time, while --schedule deploys it to run on a recurring interval.
“Snowflake CLI is the preferred command-line client for new installs and new work.”Source: docs.snowflake.com
Practise it for real
Deploy, run and schedule the Snowflake end-to-end ML task graph sample, then inspect it in Snowsight
1.From the e2e_task_graph folder, run python src/pipeline_local.py --no-register
Why: Runs the modular pipeline without orchestration, so step failures show up before a graph is involved
You should see: The pipeline completes, and the training ML Job appears in the Snowsight Job UI
2.Run python src/pipeline_dag.py --run-dag
Why: Deploys the same steps as a task graph and runs it once
You should see: The script reports execution progress and prints the final status
3.Open Monitoring > Task History in Snowsight and select the DAG
Why: Shows which tasks ran, which branch the quality check chose, and the logs of any failure
You should see: Each task's status for the latest run, with either the promotion or the alert branch taken
4.Run python src/pipeline_dag.py --schedule 1d
Why: Deploys the graph to run on a recurring schedule
You should see: The graph is deployed to run daily
Stuck? Get a nudge
If training fails on node capacity, check that the compute pool's MAX_NODES is at least the job's target_instances.
Sources
(7)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“Operationalize your ML pipeline into a Directed Acyclic Graph (DAG) and configure it to run on a schedule or by event based triggers.”
↩︎ From experiment to an automated ML pipeline“each major step (data preparation, feature engineering, model training, evaluation) becomes a separate function with clear inputs and outputs”
↩︎ Restructuring notebook code into modular steps“Parameterize all configuration values like table names and hyperparameters to enable cross-environment deployment.”
↩︎ Restructuring notebook code into modular steps“We recommend also authoring an entrypoint script which executes the end-to-end pipeline locally for debugging and future development.”
↩︎ Restructuring notebook code into modular steps“Certain distributed ML APIs are only available inside the Container Runtime, and attempting to import them outside the Container Runtime environment will fail.”
↩︎ Running each step as an ML Job - 2.
“The dependencies must run in a start-to-finish direction, with no loops.”
↩︎ From experiment to an automated ML pipeline“All tasks in a task graph must have the same task owner and be stored in the same database and schema.”
↩︎ From experiment to an automated ML pipeline“By default, if a child task fails, the entire task graph is considered to have failed.”
↩︎ Gating training on a validation step“When a task has multiple parents, the task waits for all preceding tasks to successfully complete before starting.”
↩︎ Gating training on a validation step“Each root task can have only one finalizer task, and a finalizer task can be associated with only one root task.”
↩︎ Finalizer tasks and overlapping runs“A finalizer task cannot have any child tasks.”
↩︎ Finalizer tasks and overlapping runs“A finalizer task is scheduled only when no other tasks are running or queued in the current task graph run.”
↩︎ Finalizer tasks and overlapping runs“OVERLAP_POLICY = NO_OVERLAP (default): Executes tasks serially with no parallelism.”
↩︎ Finalizer tasks and overlapping runs“Resume all of the tasks in a task graph at once using SYSTEM$TASK_DEPENDENTS_ENABLE ( <root_task_name> ) on the root task.”
↩︎ Starting, running and retrying a task graph“To retry the latest graph run, use EXECUTE TASK … RETRY LAST to attempt to run the task graph from the last failed task.”
↩︎ Starting, running and retrying a task graph“By default, a task graph is suspended after 10 consecutive failures.”
↩︎ Starting, running and retrying a task graph“When you suspend a child task, the task graph continues to run as though the child task had succeeded.”
↩︎ Starting, running and retrying a task graph“You can create tasks and task graphs using supported languages and tools like SQL, JavaScript, Python, Java, Scala, or Snowflake Scripting.”
↩︎ Driving pipelines from Snowflake CLI, SnowSQL and the SDKs“When a task has multiple parents, the task waits for all preceding tasks to successfully complete before starting.”
↩︎ Key concept“specifying logic-based operations in the task body using runtime values, graph level configuration, and return values of parent tasks”
↩︎ Exam trap 1“When the root task of a task graph is skipped (for example, because of overlap task graph runs), the finalizer task won’t be started.”
↩︎ Exam trap 2“When you run the root task, all resumed child tasks in the task graph are executed in the order defined by the task graph.”
↩︎ Exam trap 3“at least one run of the task graph is skipped”
↩︎ Exam trap 4“Task graphs can run on a recurring schedule, or they can be triggered by an event.”
↩︎ Checkpoint“OVERLAP_POLICY = ALLOW_ALL_OVERLAP: Allows unlimited true parallelism. Snowflake can run multiple instances of the entire task graph, including the root task, concurrently.”
↩︎ Checkpoint - 3.
“returns the payload’s return value or, if execution failed, raises an exception”
↩︎ Running each step as an ML Job“For file-based jobs, use the special __return__ variable to specify the return value.”
↩︎ Running each step as an ML Job“Snowflake automatically uses the latest available version of the Snowflake Container Runtime on your compute pool”
↩︎ Running each step as an ML Job“You can run them from any development environment.”
↩︎ Driving pipelines from Snowflake CLI, SnowSQL and the SDKs“then returns the payload’s return value or, if execution failed, raises an exception.”
↩︎ Checkpoint - 4.https://www.snowflake.com/en/developers/guides/e2e-task-graphSecondary source
“Branching Logic: Using DAGTaskBranch for conditional execution paths”
↩︎ Gating training on a validation step“Task Context: Passing values between Tasks in the DAG via the TaskContext”
↩︎ Passing values between tasks and triggering on new data - 5.
“In a task graph, a task can call this function to set a return value.”
↩︎ Passing values between tasks and triggering on new data“The string size must be <= 10 kB (when encoded in UTF8).”
↩︎ Passing values between tasks and triggering on new data - 6.
“Define the target stream using the WHEN clause. (Do not include the SCHEDULE parameter.)”
↩︎ Passing values between tasks and triggering on new data“Triggered tasks don’t use compute resources until the event is triggered.”
↩︎ Passing values between tasks and triggering on new data - 7.
“Snowflake CLI is the preferred command-line client for new installs and new work.”
↩︎ Driving pipelines from Snowflake CLI, SnowSQL and the SDKs“Snowflake supports SnowSQL 1.5.x through April 16, 2028.”
↩︎ Driving pipelines from Snowflake CLI, SnowSQL and the SDKs