What you will be able to do
- Find job and pipeline runs in the Runs tab and filter them by status, error code, and time range
- Use the Finished runs count graph and Top 5 error types table to judge failure rates
- Tell Succeeded, Succeeded with failures, Failed, Skipped, Timed Out, and Canceled apart using the leaf-task rule
- Read a job's matrix view to compare a run's duration and status with earlier runs
Key concept
Leaf-task outcome rule — Lakeflow Jobs decides whether a whole job run succeeded by looking only at its leaf tasks, which are tasks that no other task depends on. A failure partway up the graph does not mark the run Failed if every leaf task still succeeds.
1.The Runs tab: a workspace-wide health view
Monitoring starts at Jobs & Pipelines in the workspace sidebar. The Jobs & pipelines tab lists every job and pipeline you can access. For each one it shows the creator, the trigger, and the result of its last five runs, so you can spot a job that keeps failing without opening it. For a closer look at health, open the Runs tab. It lists running and recently completed runs for every job and pipeline you can access, including runs started by external orchestrators such as Apache Airflow or Azure Data Factory.
The Runs tab has three parts, and the filters at the top apply to all of them. You can filter by job or pipeline name, by type (All, Jobs, or Pipelines), by pipeline type, by Run as user, by start or end time, by run status, by run ID, and by error code. For a time filter, completed runs are matched on their end time and in-progress runs on their start time.
The Finished runs count graph gives you the failure rate. By default it shows failed, skipped, and successful runs from the last 48 hours. You can narrow it to particular statuses, or click and drag across it to zoom into a time window. The graph and the table then update to that window. The graph has two visibility rules. It appears only when the type filter is Jobs or Pipelines, not All. Admins see it for all runs, but non-admins must set Run as to me before it appears.
The Top 5 error types table lists the most frequent errors in the selected range, so recurring failures stand out. The runs table holds 60 days of history. Each row shows the start time, the name, the type, the Run as user, what launched the run (schedule, API request, or manual), the duration, the status, any error code, and the parameters. If a duration passes a configured expected completion time, the UI shows a warning.
| Component | Window | Use it to |
|---|---|---|
| Finished runs count graph | Last 48 hours (narrow with time filter or drag) | See how many runs failed, were skipped, or succeeded |
| Top 5 error types | Selected time range | Find the most common causes of failure |
| Runs table | Last 60 days | Inspect individual runs: status, duration, error code, trigger, parameters |
Checkpoint 1 of 5· Check yourself
A non-admin engineer opens the Runs tab with the type filter set to Jobs, but the Finished runs count graph does not appear. What should they do?
Admins see the graph for all runs. Non-admins only see it after filtering Run as to themselves. Switching the type filter to All would hide the graph entirely.
“For non-admins, you must click Run as and select me.”Source: docs.databricks.com
Sources1
2.Interpreting job run statuses
A run in the list has one of these statuses: Queued, Pending, Running, Skipped, Succeeded, Succeeded with failures, Failed, Timed Out, Canceling, or Canceled. Queued, Pending, Running, and Canceling describe runs that are still in progress. The others are final outcomes, and you need to know what each one means.
The key rule is that a run's outcome depends on its leaf tasks, which are tasks with no downstream dependencies. Tasks in the middle of the graph do not decide the outcome on their own.
| Outcome | Meaning |
|---|---|
| Succeeded | All tasks were successful |
| Succeeded with failures | Some tasks failed, but all leaf tasks were successful |
| Failed | One or more leaf tasks failed |
| Skipped | The run was skipped, for example because maximum concurrent runs for the job or workspace was exceeded |
| Timed Out | The run took too long to complete and timed out |
| Canceled | The run was canceled, for example manually by a user |
Two of these outcomes are easy to misread. Succeeded with failures looks like a pass, but something in the graph broke, and that task's output may be missing even though the final tasks finished. Skipped usually points to scheduling pressure rather than bad code. The documented example is a run that exceeded the maximum concurrent runs for the job or the workspace. Individual tasks can also end as Disabled, either because you disabled them in the job settings or because an upstream task was disabled.
Checkpoint 2 of 5· Check yourself
Every task in a job run succeeds except one leaf task, which fails. What is the run's status?
A failed leaf task always makes the run Failed, however many other tasks succeeded.
“Failed: One or more leaf tasks failed.”Source: docs.databricks.com
Checkpoint 3 of 5· Exam question
In the Lakeflow Jobs UI Graph view for a nightly job, the ingest task shows a failed status and both downstream transform tasks show a skipped status instead of failed or running. What does this indicate, and what should the data engineer check first?
Correct answer: B — The transform tasks never ran because their default run-if condition, All succeeded, was not met after ingest failed; the engineer should open the ingest task's run details first.
- A. A manually disabled task is shown with its own explicit disabled indicator in the task settings and graph, which is a different signal from a run-time skip caused by a failed upstream dependency. Re-enabling tasks that were never disabled would not address the actual failure.
- B. Tasks use the All succeeded run-if condition by default, so when an upstream task fails its downstream tasks never start and are marked skipped rather than failed. Tracing the root cause always starts at the task that actually failed, since the skipped tasks have no error of their own to investigate.
- C. A skipped status specifically means the task never executed at all, so there are no driver logs to review for it. This option also ignores the direct dependency link visible in the graph between the failed ingest task and the tasks after it.
- D. A queued retry is shown with a distinct pending or running indicator, not a skipped status, so this misreads what the graph is displaying. Raising the retry count would not change anything, since the skip is caused by the failed dependency, not by a transient cluster problem.
3.The matrix view: comparing a run against its history
The workspace Runs tab shows many jobs side by side. To follow one job over time, click its name. The job's own Runs tab has a matrix view and a list view. The matrix view shows the job's run history, including each task in each run, so you can compare the latest run with earlier ones.
The matrix has two kinds of rows. The Run total duration row has one bar per run. Hover over a bar to see the run's start time, duration, and status. The Tasks rows have one cell per task. Hover over a cell to see the task's start time, duration, cluster, and status. Two visual cues carry most of the information. Colour shows the status, and the height of each run or task bar shows its duration. A bar much taller than the ones beside it is a slow run that you can spot without reading any numbers.
Checkpoint 4 of 5· Match them up
Match each bar colour in the matrix view to the status it shows
Tap a term, then the definition that fits it.
Job run bars and task run bars use the same colour code, so a red or pink cell shows which task in which run had a problem.
“Successful runs are green. Unsuccessful runs are red, skipped runs are pink, and waiting for retry is yellow.”Source: docs.databricks.com
If you configure an expected completion time, you no longer have to judge bar heights by eye. When a run takes longer than that time, the matrix view and the runs list both show a warning. To compare a failing run with a known-good baseline, click Go to the latest successful run. Click the start time of any run to open its job run details page.
Run history has a time limit. Databricks keeps job run history for up to 60 days and recommends exporting results you need to keep before they expire. A long-term failure-rate trend therefore has to come from somewhere else. If you have access to the system.lakeflow schema, you can query job run and task records across the account.
Checkpoint 5 of 5· Check yourself
In a job's matrix view, one run's bar is about three times taller than the bars for the runs on either side. What does the height tell you?
Bar height shows duration. Status is shown by colour, not height.
“The height of the individual job run and task run bars visually indicates the run duration.”Source: docs.databricks.com
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.The Finished runs count graph shows the same 60-day window as the runs table.Why is that wrong?
The graph covers runs completed in the last 48 hours. The runs table holds 60 days of history.
Covered in The Runs tab: a workspace-wide health view
2.A job run is marked Failed whenever any of its tasks fails.Why is that wrong?
Only leaf-task failures make a run Failed. If non-leaf tasks fail but every leaf task succeeds, the run is Succeeded with failures.
Covered in Interpreting job run statuses
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.https://docs.databricks.com/aws/en/jobs/monitorOfficial docs
“such as the creator, the trigger (if any), and the result of the last five runs.”
↩︎ The Runs tab: a workspace-wide health view“including runs started by external orchestration tools such as Apache Airflow or Azure Data Factory”
↩︎ The Runs tab: a workspace-wide health view“The selected time range is matched against each run's end time for completed runs, or its start time for in-progress runs.”
↩︎ The Runs tab: a workspace-wide health view“The Finished runs count graph appears only when you filter to Jobs or Pipelines.”
↩︎ The Runs tab: a workspace-wide health view“The Top 5 error types table shows a list of the most frequent error types from the selected time range”
↩︎ The Runs tab: a workspace-wide health view“The UI displays a warning if the duration exceeds a configured expected completion time.”
↩︎ The Runs tab: a workspace-wide health view“The status of the run: Queued, Pending, Running, Skipped, Succeeded, Succeeded with failures, Failed, Timed Out, Canceling, or Canceled.”
↩︎ Interpreting job run statuses“a task might be skipped because you exceeded the maximum concurrent runs for your job or your workspace”
↩︎ Interpreting job run statuses“Individual tasks can also terminate with a Disabled status when you explicitly disable them in the job settings”
↩︎ Interpreting job run statuses“The matrix view shows a history of runs for the job, including each job task.”
↩︎ The matrix view: comparing a run against its history“the matrix view displays a warning when the duration of a run exceeds the configured time.”
↩︎ The matrix view: comparing a run against its history“To view details for this job's most recent successful run, click Go to the latest successful run.”
↩︎ The matrix view: comparing a run against its history“Pending, canceled, or timed out are grey.”
↩︎ The matrix view: comparing a run against its history“Databricks determines whether a job run was successful based on the outcome of the job's leaf tasks.”
↩︎ Key concept“The Finished runs count graph shows the number of runs completed in the last 48 hours.”
↩︎ Exam trap 1“Succeeded with failures: Some tasks failed, but all leaf tasks were successful.”
↩︎ Exam trap 2“The Finished runs count graph shows the number of runs completed in the last 48 hours.”
↩︎ Prediction“For non-admins, you must click Run as and select me.”
↩︎ Checkpoint“Succeeded with failures: Some tasks failed, but all leaf tasks were successful.”
↩︎ Prediction“Failed: One or more leaf tasks failed.”
↩︎ Checkpoint“Successful runs are green. Unsuccessful runs are red, skipped runs are pink, and waiting for retry is yellow.”
↩︎ Checkpoint“The height of the individual job run and task run bars visually indicates the run duration.”
↩︎ Checkpoint - 2.
“A task run may be unsuccessful because it failed or was skipped because a dependent task failed.”
↩︎ Interpreting job run statuses“The matrix view in the Runs tab shows a history of runs for the job, including successful and unsuccessful runs for each job task.”
↩︎ The matrix view: comparing a run against its history