Exponentiating Log-Transformed Targets Before Metrics and Interpretation

1.A log-target model predicts logs, not prices

Regression targets such as house prices, revenue, salaries or transaction amounts are usually right-skewed: most values are modest and a few are huge. A common fix is to train on the logarithm of the label instead of the raw value. The log pulls in the long tail, makes the residuals more symmetric, and stops a handful of extreme rows from dominating a squared-error loss. In Databricks SQL and PySpark this usually means ln (or log with a single argument, which is the same thing), applied to the label column before training.

The catch is simple, but the exam keeps testing it. The model has no idea the column used to be a price. It learned to predict whatever numbers you gave it, so if the label was ln(price), then predict returns ln(price). For a house that sells for about $200,000, the natural log is about 12.2, and 12.2 is what comes out of the model. Nothing in the training loop converts it back for you. The exception is a wrapper built for this job, such as scikit-learn's TransformedTargetRegressor, which applies the inverse automatically.

Sources123

2.Undoing the transform: match the inverse to the forward function

To exponentiate correctly, you need to know exactly which log was applied, because each forward function has its own inverse. The natural log ln(x) is undone by exp(x), which is e raised to the prediction. Many pipelines use log1p(x) instead, which is ln(1 + x). It exists because a plain log fails on zero: Databricks returns NULL when the argument is less than or equal to 0, and labels such as sales or counts often contain zeros. The inverse of log1p is expm1, which is e^x − 1. If you apply plain exp to a log1p prediction, every value comes out one unit too high. That error is easy to miss when values are large and very visible when they are near zero.

from pyspark.sql import functions as dbf
df = spark.sql("SELECT id AS value FROM RANGE(5)")
df.select("*", dbf.expm1(df.value)).show()

from pyspark.sql import functions as dbf
df = spark.sql("SELECT id AS value FROM RANGE(5)")
df.select("*", dbf. ? (df.value)).show()

Sources425

3.Compute evaluation metrics after exponentiating, not before

Regression metrics such as RMSE and MAE are built from the difference between a prediction and a true label. The Databricks glm tutorial below shows this directly: the error column is price - prediction, and RMSE is the square root of the mean squared error. That number is in dollars only because both columns are in dollars. If the model had been trained on ln(price), its prediction column would hold log values. Subtracting those from raw prices produces meaningless numbers, and subtracting them from logged prices produces an error in log units, which is not what a business stakeholder or an exam question asking for RMSE in the original units wants.

errors <- select(predictions, predictions$price, predictions$prediction, alias(predictions$price - predictions$prediction, "error"))
display(errors)

# Calculate RMSE
head(select(errors, alias(sqrt(sum(errors$error^2 , na.rm = TRUE) / nrow(errors)), "RMSE")))

So the order of operations is fixed. Transform the label, train, predict (getting log values), exponentiate the predictions, and only then compute the metric against the original, untransformed labels. Do not take a shortcut by computing RMSE on the log scale and exponentiating the result. RMSE is a nonlinear summary over all rows, so exp(RMSE_log) is not the dollar RMSE. The exponentiation has to happen row by row, on the predictions, before any averaging. scikit-learn's TransformedTargetRegressor builds this into the estimator: it fits on the transformed target and maps predictions back with the inverse transform, so predict and score already work in the original units.

Sources63

4.Interpreting log-target predictions and effects

The same rule applies when you explain a model, not just when you score it. A prediction of 12.2 means nothing to a pricing team, while about $199,000 does. Every number that leaves the notebook (a dashboard value, a forecast in a report, a threshold for an alert) should be exponentiated with the inverse that matches the forward transform. In Databricks SQL that is exp, which returns e to the power of its argument, and in PySpark it is dbf.exp (or dbf.expm1 for log1p labels).

Effects change shape as well. With a log-transformed label, a linear model's coefficient no longer means 'adds this many dollars'. It acts multiplicatively on the original scale: a one-unit increase in a feature multiplies the predicted value by exp(coefficient). For small coefficients this is roughly a percentage change of 100 × coefficient. Reading a log-scale coefficient as an additive dollar amount is the interpretation version of the same mistake.

Sources74

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.An RMSE computed between log predictions and log labels can be reported as the model's error in dollars.Why is that wrong?

   That RMSE is in log units. A dollar RMSE needs predictions mapped back to the original space and compared with the original labels.

   Covered in Compute evaluation metrics after exponentiating, not before

2. 2.You can compute RMSE on the log scale and then exponentiate the RMSE to get the original-scale error.Why is that wrong?

   The metric must measure the distance between original-scale predictions and original labels, so exponentiate each prediction first and then compute the metric.

   Covered in Compute evaluation metrics after exponentiating, not before

3. 3.Plain exp correctly inverts any log transform, including log1p.Why is that wrong?

   log1p is ln(1 + x), so its inverse is expm1 (the exponential minus one). Using exp leaves every prediction one unit too high.

   Covered in Undoing the transform: match the inverse to the forward function