What you will be able to do
- Open a job run's graph, timeline, and list views and say what each one is for
- Trace a failed or skipped task back to the upstream task that blocked it
- Interpret the Upstream failed and Excluded task states produced by Run if conditions
- Use the run breakdown and timeline view to find where a run's time went
1.Three views of a single job run
A job's run history shows that a run went wrong. The job run details page shows where in the run it went wrong. To open it, click Jobs & Pipelines in the sidebar, click the job's name, and then click the start time of a run. The page contains the run's output and links to logs, and it reports whether each task succeeded or failed.
Jobs with more than one task get three views of the same run:
- Graph: the DAG of tasks. Click a task node to see its run details: the Run as user, how it was launched, its start time, end time, duration, and status, plus its source code, the cluster it ran on (with links to query history and logs), and task metrics. - Timeline: shows which tasks took a long time and how tasks depended on and overlapped one another. - List: one row per task. By default it shows status, name, type, resource, duration, and dependencies. You can search by name, filter by status or type, and sort by status, name, or duration.
Checkpoint 1 of 5· Check yourself
A 40-task job ran slowly and you want to know which tasks took the longest. Which view, used as described in the documentation, ranks them most directly?
The list view can sort tasks by duration. The graph would mean clicking 40 nodes one at a time, and the error views only cover failures.
“You can search for a task by name, filter by task status or task type, and sort tasks by status, name, or duration.”Source: docs.databricks.com
Sources1
2.Reading the DAG to find upstream blockers
Each line in the job DAG is a dependency. Databricks runs upstream tasks before downstream ones and runs as many tasks in parallel as it can. This means one failure near the top of the graph can stop everything below it. A task can be unsuccessful because it failed itself, or because it was skipped after a task it depends on failed. When you see a red or skipped task, first check whether it actually caused the problem or was only affected by it.
What happens downstream depends on each task's Run if condition. The default is All succeeded.
| Run if condition | Runs when | State if unmet |
|---|---|---|
| All succeeded (default) | All dependencies ran and succeeded | Upstream failed |
| At least one succeeded | At least one dependency succeeded | Upstream failed |
| None failed | No dependency failed and at least one ran | Upstream failed |
| All done | All dependencies have run, whatever their outcome | Not applicable: always runs |
| At least one failed | At least one dependency failed | Excluded |
| All failed | All dependencies failed | Excluded |
When you trace a blocker back through the graph, three rules matter:
1. Upstream failed and Upstream canceled count as *failed*, not skipped, when a further downstream task checks its own Run if condition. The failure keeps propagating down the graph. 2. Excluded counts as *successful*. Tasks marked Excluded are usually failure handlers whose condition was not met, and an excluded task is skipped. If all of a task's dependencies are excluded, that task is excluded too, so exclusion cascades along a linear chain. 3. Cancellation propagates. If you cancel a task run, the cancellation passes to downstream tasks, but tasks whose Run if handles failure still run. This lets a cleanup task run after a cancellation.
The blocker is not always inside the job. Sometimes the cause is upstream of the job itself, for example an external data source that is unavailable.
Checkpoint 2 of 5· Put it in order
You have been told that a job run failed. Put these steps for finding the cause in the Lakeflow Jobs UI in order.
- 1.Click the job name to open its Runs tab and matrix view
- 2.Hover over the failed task to see its metadata
- 3.Click the failed task to open the Task run details page with its output and error message
- 4.Click Jobs & Pipelines in the sidebar
You go from the job list to that job's run history. You hover over the failed cell for a quick look, then click it to see the full output and error.
“To help identify the cause of the failure, click the failed task.”Source: docs.databricks.com
Checkpoint 3 of 5· Exam question
A data engineer wants to quickly see whether a job's failure rate has been increasing over the past two weeks without opening each individual run. Which Lakeflow Jobs UI view is best suited to this, and why?
Correct answer: C — The Matrix view is best because it renders each run as a color-coded bar across a timeline, so red failure bars and their frequency stand out immediately.
- A. A chronological runs list does show a status per row, but scanning dozens of rows to judge a trend is slower and more error-prone than a single visual summary of colored bars. It also does not aggregate the rate for the engineer automatically.
- B. The graph view represents the dependency structure of one specific run, not a history of many runs over time, so it cannot show a two-week trend on its own. It is useful for diagnosing a single failure, not for spotting a rate change.
- C. The matrix layout plots many runs side by side with color-coded bars, so a rising trend of red failure bars over two weeks is visible at a glance. This is exactly the kind of high-level pattern spotting the view is designed for, unlike views built around a single run's detail.
- D. The timeline is built to show task-level overlap and duration within one run, not to summarize success or failure across many runs. It would require opening each run individually, which defeats the goal of a quick trend check.
3.Tracking where a run's time went
A run can be slow without failing. The Jobs UI gives you two tools to find out why. The timeline view shows which tasks ran long and where tasks overlapped. For serverless jobs, the timeline also includes each task's queries and links to their query profiles. The run breakdown splits the run's total time into phases. On the job run details page, hover over the run's Duration to see it.
| Phase | A large value suggests |
|---|---|
| Queued | Waiting because of concurrency limits: the job's maximum concurrent runs or the workspace-level limit |
| Waiting for resources | Waiting for compute to become available |
| Library installation | Time spent installing libraries before the run starts |
| Running | The work itself: more data, configuration changes, or slower code |
The breakdown is available for both job runs and task runs. In a multi-task job, the job-level breakdown counts Waiting for resources only until the first task starts. For the full picture of each task, open that task's own breakdown. If Running is the largest phase, the run may simply have processed more data, which can be expected. To check for configuration or code changes, review recent changes in the jobs and compute system tables.
Duration also needs care when a run has been repaired. In the Runs tab, a repaired run's Duration covers the time from the first attempt's start to the latest repair's finish. A run that failed twice and then succeeded therefore shows the combined time of all three attempts. Do not take that number as the time one run takes.
Checkpoint 4 of 5· Check yourself
A multi-task job run failed, was repaired, and then succeeded. Its Duration in the Runs tab is much higher than usual. What is the most likely explanation?
After a repair, the Runs tab Duration spans every attempt. It is not a regression in any single attempt.
“includes the time the first run started until the time when the latest repair run finished.”Source: docs.databricks.com
Checkpoint 5 of 5· Exam question
A pipeline job that normally completes in about 12 minutes has been taking over 40 minutes for the last three runs, though it still finishes with a Succeeded status each time. Using only the Lakeflow Jobs UI, how can the engineer confirm this is an unusual trend rather than normal variance?
Correct answer: A — Open the job's run history and compare the total duration bars across recent runs against the historical baseline shown in the Matrix or Runs list view.
- A. The run history views display each run's total duration alongside prior runs, so lining up recent bars against the historical pattern is the direct way to confirm a 12-minute job is now consistently taking 40 minutes. This does not require leaving the Jobs UI at all.
- B. Stage-level shuffle metrics can help explain why a run is slow, but they are not needed just to confirm that a duration trend exists, and reading only the most recent run would miss the comparison against the historical baseline entirely.
- C. The Jobs UI does report run duration for every run, so this understates what the UI provides and sends the engineer to a source that does not track run-over-run trends the way the run history view does.
- D. Repair run reruns only the tasks that failed in a specific past run, and since every recent run already succeeded there is nothing to repair; it also would not establish a historical duration baseline.
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.A downstream task marked Upstream failed is treated like a skipped task, so tasks further downstream can still run normally.Why is that wrong?
Upstream failed and Upstream canceled count as failed when Run if conditions are checked, so the failure keeps propagating.
Covered in Reading the DAG to find upstream blockers
2.Every red or skipped task in a run has its own error to investigate.Why is that wrong?
A task can be unsuccessful only because a task it depends on failed. Trace back through the DAG to the task that failed first.
Covered in Reading the DAG to find upstream blockers
3.A repaired run's Duration in the Runs tab is the time taken by the final, successful attempt.Why is that wrong?
Duration runs from the first attempt's start to the latest repair's finish, so it includes every attempt.
Covered in Tracking where a run's time went
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.https://docs.databricks.com/aws/en/jobs/monitorOfficial docs
“Jobs with multiple tasks additionally have a graph, timeline, and list view.”
↩︎ Three views of a single job run“Click a task node in the graph to view task run details, including:”
↩︎ Three views of a single job run“By default, the list view shows the status, name, type, resource, duration, and dependencies.”
↩︎ Three views of a single job run“Jobs that contain multiple tasks have a timeline view to identify tasks that are taking a long time to complete”
↩︎ Tracking where a run's time went“You can search for a task by name, filter by task status or task type, and sort tasks by status, name, or duration.”
↩︎ Checkpoint - 2.https://docs.databricks.com/aws/en/jobs/run-ifOfficial docs
“Dependencies are visually represented in the job DAG as lines between tasks.”
↩︎ Reading the DAG to find upstream blockers“Databricks runs upstream tasks before running downstream tasks, running as many of them in parallel as possible.”
↩︎ Reading the DAG to find upstream blockers“Upstream tasks with the Excluded result state are treated as successful when evaluating Run if conditions.”
↩︎ Reading the DAG to find upstream blockers“This exclusion cascades along a linear dependency chain.”
↩︎ Reading the DAG to find upstream blockers“If you cancel a task run, the cancellation propagates through downstream tasks”
↩︎ Reading the DAG to find upstream blockers“Upstream tasks marked Upstream failed or Upstream canceled are treated as failed, not as skipped.”
↩︎ Exam trap 1“This is the default setting. The task is marked as Upstream failed if the condition is unmet.”
↩︎ Prediction - 3.
“In some cases, the cause of a failure may be upstream from your job; for example, an external data source is unavailable.”
↩︎ Reading the DAG to find upstream blockers“Hover over a failed task to see associated metadata.”
↩︎ Reading the DAG to find upstream blockers“A task run may be unsuccessful because it failed or was skipped because a dependent task failed.”
↩︎ Exam trap 2“includes the time the first run started until the time when the latest repair run finished.”
↩︎ Exam trap 3“To help identify the cause of the failure, click the failed task.”
↩︎ Checkpoint“includes the time the first run started until the time when the latest repair run finished.”
↩︎ Checkpoint - 4.
“hover over the run's Duration to see the total duration broken down into the following phases:”
↩︎ Tracking where a run's time went“the job-level breakdown counts Waiting for resources only until the first task starts running.”
↩︎ Tracking where a run's time went“Your job or task is waiting to start because of concurrency limits.”
↩︎ Tracking where a run's time went“Data volume increased, so a longer runtime might be expected.”
↩︎ Tracking where a run's time went“Review recent changes using Databricks system tables, such as the jobs system tables and compute system tables.”
↩︎ Tracking where a run's time went