What you will be able to do
- Write run search queries on parameters, metrics and tags, and know which metric value they filter on
- Compare selected runs side by side, within one experiment or across several
- Use chart view and parallel coordinates plots to relate parameters to metrics
- Download runs as CSV and find deleted runs, knowing the limits of each
1.Searching the runs table
An experiment with hundreds of runs is only useful if you can find the right one. The runs table on the experiment details page has a search field that takes expressions over parameter values, metric values and tags. Type a query and press Enter. Parameters are written as params.<name>, metrics as metrics.<name>, and tags as tags.<key>.
| Query | Matches runs where… |
|---|---|
| metrics.r2 > 0.3 | the last logged r2 is above 0.3 |
| params.elasticNetParam = 0.5 AND metrics.avg_areaUnderROC > 0.3 | both the parameter and the metric conditions hold |
| MIN(metrics.rmse) <= 1 | the lowest rmse ever logged is at most 1 |
| MAX(metrics.memUsage) > 0.9 | the highest memUsage ever logged exceeds 0.9 |
| tags.estimator_name="RandomForestRegressor" | the tag has that string value (quotes required) |
tags.my custom tag = "my value" | a tag whose key contains spaces (backticks required) |
There are two catches. MIN and MAX only work for runs logged after August 2024, because older runs have no stored minimum and maximum. And tag string values must be in quotes. Besides the search field, the Time created, State (Active or Deleted) and Datasets drop-downs filter runs by creation time, state and the datasets used. The Columns control sets which columns the runs table shows.
Checkpoint 1 of 5· Check yourself
Which query finds runs whose best-ever rmse is at most 1, even if their final rmse is higher?
MIN filters on the lowest logged value. A bare metrics.rmse, like LATEST, uses the last logged value, and rmse is a metric, not a param.
“Using MIN or MAX lets you search for runs based on the minimum or maximum metric values, respectively.”Source: docs.databricks.com
Sources1
2.Comparing runs side by side
Search narrows the list. Comparing tells you why the remaining runs differ. Select two or more runs with the checkboxes on the experiment details page and click Compare. The Comparing <N> Runs screen shows the selected runs in tables: a Run details table at the top, then Parameters and Metrics tables with one column per run. A toggle hides parameters and metrics that are the same in every selected run, so only the settings that actually changed are left.
You can also compare across experiments. Start from the Experiments page instead of a single experiment.
Checkpoint 2 of 5· Put it in order
Put the steps for comparing runs from several experiments in order.
- 1.Click Compare (n) to see all runs from those experiments
- 2.Click Compare to open the Comparing <N> Runs screen
- 3.On the experiments page, check the boxes next to the experiments you want
- 4.Select two or more runs with their checkboxes
You first pick experiments, which brings up a combined run list. Then you pick runs from that list and compare them.
“Click Compare (n) (n is the number of experiments you selected). A screen appears showing all of the runs from the experiments you selected.”Source: docs.databricks.com
Checkpoint 3 of 5· Exam question
During training, a script calls `mlflow.log_metric("val_loss", value, step=epoch)` once per epoch across 40 epochs. After the run finishes, the data scientist wants to see whether `val_loss` decreased steadily or plateaued partway through training. What does the run's page in the MLflow UI provide for this?
Correct answer: A — The Metrics tab for that run renders `val_loss` as a line chart plotted against the logged `step` values, so the shape of the loss curve across all 40 training epochs is visible directly on that page.
- A. When a metric is logged multiple times with a `step` argument, MLflow stores the full history and the run page's Metrics tab renders it as a line chart of value versus step, showing exactly the kind of plateau or steady-decrease pattern being checked here.
- B. The run overview does show the latest value for a quick reference, but the full per-step history is still tracked and chartable in the Metrics tab, so re-running training to recover the trend is unnecessary.
- C. Step-logged metrics are charted by MLflow automatically without any extra artifact logging; a manually logged matplotlib image is a separate, optional artifact and not required to see the built-in metric chart.
- D. The Tags tab stores user- or system-set key-value labels on the run, not a chronological log of metric calls, so it would not list epoch-by-epoch loss values at all.
Sources1
3.Chart view and parallel coordinates
The comparison tables show numbers. Chart view shows them as graphs. Click the Chart view icon on the experiment details page to open a set of charts comparing the experiment's runs. By default the charts show only the 10 most recent runs. You can change how many runs are shown, and runs left off the charts have a grayed-out dot in the run list. Hover over a line to see that run's details. You can move, resize, enlarge, edit, delete or download each chart.
The controls work like the runs table. The same search field filters by parameters, metrics or tags. Sort orders runs by a parameter, and Group by groups runs by one or more parameter values. Add chart creates new charts. The most useful one for tuning is the parallel coordinates plot. You choose the parameters and metrics to plot, and patterns show up across runs. In the docs' example, lower max_depth values go with higher auc.
Model answer: Chart view shows only the 10 most recent runs by default, so 30 of the 40 runs are grayed out. Increase the number of runs displayed with the control at the top of the run list. Then use Add chart and choose Parallel coordinates, selecting your three parameters and the validation metric, to see how they relate across runs.
Checkpoint 4 of 5· Check yourself
Which chart type is best for seeing how several parameter settings together relate to a metric such as auc?
A parallel coordinates plot puts the parameters and metrics you choose on parallel axes, so you can see relationships across runs.
“A parallel coordinates plot is useful in understanding the effect of parameter settings on model performance”Source: docs.databricks.com
Sources2
4.Exporting and deleting runs from the UI
The experiment details page can also export what it shows. In its kebab menu, Download <n> runs creates a CSV with one row per run. The columns are Start Time, Duration, Run ID, Name, Source Type, Source Name, User and Status, then one column per parameter and per metric. This download covers the runs shown and is capped at 100 of them, so an experiment with more runs gives you at most 100. For more than 100, choose Download all runs instead. That opens a code snippet you run in a notebook, and you download the rows from the cell output.
To delete runs, select them with their checkboxes and click Delete. For a parent run, you also choose whether to delete its child runs, which is selected by default. Deleted runs are kept for 30 days. Choose Deleted in the State filter to see them.
With MLflow 3, models logged from runs also appear on the experiment's Models tab. There you can search and filter Logged Models by attributes, parameters, tags and metrics. The filterable attributes are model_id, model_name, status, artifact_uri, creation_time and last_updated_time. The tab's Charts tab provides visualizations to help you compare models.
Checkpoint 5 of 5· Check yourself
An experiment shows 450 runs. You choose Download <n> runs from the kebab menu. What do you get?
The direct CSV download covers the runs shown, up to a maximum of 100. Download all runs gives you a notebook snippet for larger or programmatic exports.
“To download a file in CSV format containing all runs shown (up to a maximum of 100), select Download <n> runs.”Source: docs.databricks.com
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.A search like metrics.rmse <= 1 matches any run that reached that rmse at some point during training.Why is that wrong?
A bare metric search compares the last logged value. Use MIN(...) or MAX(...) to filter on the extreme values.
Covered in Searching the runs table
2.Chart view plots every run in the experiment by default.Why is that wrong?
Charts show only the 10 most recent runs unless you change the number of runs displayed.
Covered in Chart view and parallel coordinates
3.Deleting a run in the UI removes it immediately and permanently.Why is that wrong?
Deleted runs are kept for 30 days. Select Deleted in the State filter to see them.
Covered in Exporting and deleting runs from the UI
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.https://docs.databricks.com/aws/en/mlflow/runsOfficial docs
“Only runs logged after August 2024 have minimum and maximum metric values.”
↩︎ Searching the runs table“If the key includes spaces, you must enclose it in backticks as shown.”
↩︎ Searching the runs table“You can also filter runs based on their state (Active or Deleted), when the run was created, and what datasets were used.”
↩︎ Searching the runs table“The Parameters and Metrics tables display the run parameters and metrics from all selected runs.”
↩︎ Comparing runs side by side“you can hide parameters and metrics that are identical in all selected runs by toggling”
↩︎ Comparing runs side by side“To display deleted runs, select Deleted in the State field.”
↩︎ Exporting and deleting runs from the UI“By default, metric values are filtered based on the last logged value.”
↩︎ Exam trap 1“Deleted runs are saved for 30 days.”
↩︎ Exam trap 3“By default, metric values are filtered based on the last logged value.”
↩︎ Prediction“Using MIN or MAX lets you search for runs based on the minimum or maximum metric values, respectively.”
↩︎ Checkpoint“Click Compare (n) (n is the number of experiments you selected). A screen appears showing all of the runs from the experiments you selected.”
↩︎ Checkpoint“To download a file in CSV format containing all runs shown (up to a maximum of 100), select Download <n> runs.”
↩︎ Checkpoint - 2.
“To group runs by parameter value, select one or more parameters from the Group by dropdown menu.”
↩︎ Chart view and parallel coordinates“To display the chart view page, click the Chart view icon on the experiment details page.”
↩︎ Chart view and parallel coordinates“To select the number of runs to display, click at the top of the list of runs.”
↩︎ Chart view and parallel coordinates“By default, charts on this page show the most recent 10 runs.”
↩︎ Exam trap 2“A parallel coordinates plot is useful in understanding the effect of parameter settings on model performance”
↩︎ Checkpoint - 3.
“From the Models tab, you can search and filter Logged Models based on their attributes, parameters, tags, and metrics.”
↩︎ Exporting and deleting runs from the UI - 4.
“The Charts tab on this page provides visualizations to help you compare models”
↩︎ Exporting and deleting runs from the UI