Provisioned Throughput and LLM Spend Attribution on Databricks

1.Pay-per-token or reserved provisioned throughput

How you buy model capacity is a cost decision in its own right. Foundation Model APIs default to pay-per-token, where you pay for each request. Reserved provisioned throughput works the other way round: you prepay for a fixed pool of model units on one foundation model for a set term, and Databricks backs that pool with dedicated capacity for the length of the reservation. More model units means more guaranteed throughput. Databricks provides a model units estimator to help you size the pool from the traffic you expect. One endpoint can hold several reservations at the same time, and each has its own model units and term-end date.

Databricks recommends reserved provisioned throughput for business-critical applications or agents that need guaranteed throughput and latency and have predictable traffic. If traffic is unpredictable, the prepaid pool may sit idle, or much of the load may spill over to pay-per-token anyway. Not every model can be reserved: Databricks decides eligibility per model, and the create flow shows only the models you can reserve. Endpoint settings also matter. Scale to zero cuts resource use to zero when an endpoint is idle, and Databricks recommends it for testing and development. It is not recommended for production, because latency is higher and capacity is not guaranteed while the endpoint is scaled to zero.

Sources12

2.Finding out what drives spend: billing and spend tables

Before you can control costs, you need to know where they come from. Unity Gateway adds extra fields to MODEL_SERVING records in system.billing.usage so that cost can be traced back. usage_metadata.ai_gateway.endpoint_name holds the model service name, usage_metadata.ai_gateway.destination_model holds the model that served the request, identity_metadata.run_by holds the user or service principal that sent it, and custom_tags holds the service tags. This applies to both real-time and batch requests that go through the gateway. These records are in DBUs. To turn them into dollars, join system.billing.list_prices. Those dollar figures are at list price, so they don't include negotiated discounts or credits.

SELECT
  usage_metadata.ai_gateway.destination_model AS destination_model,
  SUM(usage_quantity) AS dbus
FROM system.billing.usage
WHERE billing_origin_product = 'MODEL_SERVING'
  AND usage_metadata.ai_gateway.endpoint_name IS NOT NULL
  AND usage_unit = 'DBU'
  AND usage_date >= current_date() - INTERVAL 30 DAYS
GROUP BY destination_model
ORDER BY dbus DESC;

External models are tracked separately. For requests routed through model provider services, Databricks estimates the dollar cost from token usage and the provider's published prices, applies any price multiplier configured on the service, and writes hourly totals to system.ai_gateway.external_model_spend. In that table, usage_unit is always USD, so you can sum usage_quantity directly with no price join. These figures are for information only and may not match the provider's invoice. If you'd rather not write SQL, the built-in usage dashboard (version 0.4 and above) has a Cost Analysis page covering both Databricks and external spend.

SELECT
  identity_metadata.run_by AS run_by,
  SUM(usage_quantity) AS usd
FROM  ? 
WHERE usage_start_time >= current_timestamp() - INTERVAL 30 DAYS
GROUP BY run_by
ORDER BY usd DESC;

Sources34

3.Attributing spend to teams and projects with tags

run_by tells you which user spent the money, but not which team or project it was for. Tags fill that gap, and there are two kinds. Service (endpoint) tags go on a model service, for example team = ml-platform, and every request through that service inherits them. Use these when each team or project has its own service. Request tags are set by the caller on each request with an HTTP header. Use these when many teams share one service.

Databricks-Ai-Gateway-Request-Tags: {"project": "chatbot", "team": "ml-platform"}

SELECT
  endpoint_tags['team'] AS team,
  endpoint_name,
  destination_model,
  COUNT(*) AS request_count,
  SUM(total_tokens) AS total_tokens
FROM system.ai_gateway.usage
WHERE endpoint_tags['team'] IS NOT NULL
GROUP BY endpoint_tags['team'], endpoint_name, destination_model
ORDER BY total_tokens DESC;

Sources4

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.Request tags are enough to attribute DBU cost for Databricks-provided models in system.billing.usage.Why is that wrong?

   Only service tags propagate to system.billing.usage. Request tags appear in system.ai_gateway.usage and in external model spend data.

   Covered in Attributing spend to teams and projects with tags

2. 2.Reserved provisioned throughput caps your bill at the cost of the reservation.Why is that wrong?

   Traffic above the reserved pool spills over to priority pay-per-token and is billed as that.

   Covered in Pay-per-token or reserved provisioned throughput