Log MLflow Models with log_model and Link Metrics to Them

1.What log_model writes

Logging a metric stores a number and logging an artifact stores a file. Logging a model stores something more structured. An MLflow Model is a standard packaging format that downstream tools understand, from Spark batch inference to REST serving. The format lets you save a model in different flavors, such as python-function, pytorch or sklearn. That is why the call is namespaced by flavor: mlflow.<model-type>.log_model(model, ...), for example mlflow.sklearn.log_model or mlflow.pytorch.log_model.

These environment files let you recreate the training environment with virtualenv (recommended) or conda. For built-in flavors MLflow can record dependencies reliably. For example, mlflow.sklearn.log_model logs the scikit-learn version. Autologging with those libraries does the same. When you train inside a run without autologging, log_model is the manual call that puts the model into the run.

Sources12

2.MLflow 3: the model as its own object

In MLflow 2.x, a logged model is stored as a run artifact and appears in the run's Artifacts tab. MLflow 3 adds a dedicated LoggedModel object, which log_model() creates with a unique ID. A LoggedModel carries its own metadata, such as parameters and metrics. It persists across the model's lifecycle and links to artifacts, metrics, parameters and the generating code. In deep learning training, MLflow creates a separate LoggedModel for each checkpoint, so you can compare checkpoints. The logging call stays the same log_model() API. In MLflow 3 you name the model with name= and can pass params= directly:

## Log the model
model_info = mlflow.sklearn.log_model(
  sk_model=lr,
  name="elasticnet",
  params={
    "alpha": 0.5,
    "l1_ratio": 0.5,
  },
  input_example = train_x
)

# Inspect the LoggedModel and its properties
logged_model = mlflow.get_logged_model(model_info.model_id)
print(logged_model.model_id, logged_model.params)

log_model returns a model_info with a model_id. Pass that ID to mlflow.log_metric or mlflow.log_metrics, and the evaluation metrics attach to the LoggedModel as well as to the run. You can also pass the dataset the metrics were computed on. To see the results, open the experiment and select the Models tab. It lists every LoggedModel in the experiment with its metrics, parameters and artifacts.

mlflow.log_metrics(
  metrics={
    "rmse": rmse,
    "r2": r2,
    "mae": mae,
  },
   ? =logged_model.model_id,
  dataset=train_dataset
)

Sources34

3.Logging what MLflow cannot infer

Some libraries can be installed with pip but have no built-in flavor. For those, use mlflow.pyfunc.log_model. MLflow infers dependencies with mlflow.models.infer_pip_requirements and writes them to requirements.txt as a model artifact. Log requirements with the exact version, for example f"nltk=={nltk.__version__}" instead of just nltk. When inference misses something, there are several ways to add to or override the model's environment:

mlflow.pyfunc.log_model(
   name=name,
   code_paths=[filename.py],
   data_path=data_path,
   conda_env=conda_env,
)

Sources2

4.Loading the model back from the run

You can check that a model was logged by loading it again. mlflow.<model-type>.load_model(modelpath) accepts several path forms. Examples are a run-relative path runs:/{run_id}/{model-path}, an MLflow 3 models:/{model_id} path and a Unity Catalog volumes path. For Python models, mlflow.pyfunc.load_model() loads any flavor as a generic Python function. You can also turn the model into a Spark UDF with mlflow.pyfunc.spark_udf for batch or streaming scoring.

model_loaded = mlflow.pyfunc.load_model(
  'runs:/{run_id}/model'.format(
    run_id=run.info.run_id
  )
)

You don't have to write this code by hand. When you log a model in a Databricks notebook, Databricks generates ready-to-copy snippets for loading it and predicting on Spark or pandas DataFrames.

Sources1

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.log_model captures every dependency, including Java, R and native system packages.Why is that wrong?

   MLflow infers only Python dependencies. You have to log non-Python dependencies yourself, for example as a dependency-list artifact.

   Covered in Logging what MLflow cannot infer

2. 2.If inference misses one library, overwrite the requirements with pip_requirements.Why is that wrong?

   Add the missing library with extra_pip_requirements. Replacing the full set with pip_requirements or conda_env is discouraged because it throws away the dependencies MLflow inferred.

   Covered in Logging what MLflow cannot infer

Practise it for real

In a Databricks notebook, train a scikit-learn classifier, manually log a test metric and a plot artifact, then load the logged model back from its run.

1. 1.Call mlflow.sklearn.autolog(), then train a GradientBoostingClassifier inside with mlflow.start_run(run_name='gradient_boost') as run:

   Why: Autologging records the model, parameters and training metrics for a run you opened yourself.

   You should see: A new run named gradient_boost appears in the notebook's Experiment Runs sidebar.

2. 2.Inside the same run, compute roc_auc_score on the test set and call mlflow.log_metric("test_auc", roc_auc).

   Why: Test-set scores are not autologged.

   You should see: test_auc is listed among the run's metrics.

3. 3.Save the ROC curve with roc_curve.figure_.savefig("roc_curve.png") and call mlflow.log_artifact("roc_curve.png").

   Why: log_artifact uploads a file from local disk, so the plot has to be written to a file first.

   You should see: roc_curve.png is in the run's Artifacts tab.

4. 4.After the run, load the model with mlflow.pyfunc.load_model('runs:/{run_id}/model'.format(run_id=run.info.run_id)) and compare its predictions with model.predict(X_test).

   Why: This shows the logged model can be loaded from the run alone.

   You should see: np.array_equal on the two prediction arrays returns True.