What you will be able to do
- Describe what the gold layer of a medallion architecture is for and who consumes it
- Distinguish tables, views, materialized views and streaming tables by how they store data and how they are kept up to date
- Choose the right Unity Catalog object for a gold dataset serving BI and analytics teams
Key concept
Compute on read vs. compute on refresh — A standard view stores nothing and runs its query each time someone reads it. A materialized view runs the query ahead of time, stores the result and refreshes it, so readers get precomputed answers. Choosing gold objects mostly comes down to deciding where that compute should happen.
1.What the gold layer is for
The medallion architecture moves data through three layers: bronze (raw), silver (validated) and gold (enriched). Each layer has its own job. Bronze ingests raw data. Silver does data cleaning and validation. Gold does dimensional modeling and aggregation. Gold also has different users. Silver mainly serves data engineers, analysts and data scientists who need record-level detail. Gold is built for business analysts and BI developers, data scientists and ML engineers, executives and decision makers, and operational teams.
The documentation describes gold as the layer that represents highly refined views of the data, built to drive analytics, dashboards, ML and applications. Gold data is often heavily aggregated and filtered to specific time periods or regions. It contains datasets that map to business functions. The docs list three properties for it: it consists of aggregated data tailored for analytics and reporting, it aligns with business logic and requirements, and it is optimized for performance in queries and dashboards.
That last property matters for the rest of this lesson. Optimizing gold-layer tables is called a best practice because these datasets are frequently queried. When the same dashboard query runs hundreds of times a day, the object you choose to hold its answer decides how fast and how expensive each run is.
In the docs' worked example, an ops catalog keeps each layer in its own schema. ops.gold holds fewer datasets than silver or bronze, such as customer_spending (average and total spend per customer), account_performance (daily performance per account) and business_summary (highly aggregated information for executive staff). Because gold models a business domain, some organizations build several gold layers, for example one each for HR, finance and IT.
Checkpoint 1 of 6· Check yourself
According to the medallion architecture, what work is characteristic of the gold layer?
Gold is where data is modeled dimensionally and aggregated for business users. Deduplication, type casting and quarantining invalid records happen in silver, and raw ingestion happens in bronze.
“Dimensional modeling and aggregation”Source: docs.databricks.com
Sources1
2.Four objects, four storage and refresh behaviours
Databricks names four core data objects for data engineering: tables, views, materialized views and streaming tables. Each one stores and refreshes data differently. Any of them can hold a gold dataset, so you need to know how they differ before you choose.
A table is a structured dataset stored in a specific location. Unless you specify otherwise, you get a Unity Catalog managed table. You can query and change it with SQL or DataFrame APIs, including INSERT, UPDATE, DELETE and MERGE INTO. Some process outside the table decides what gets written to it.
A view is a virtual table defined by a query. It stores no data. Views simplify complex queries, encapsulate business logic and give consumers a consistent interface to the underlying data without duplicating storage.
A materialized view is also defined by a query, but it precomputes and stores the result. Queries against it can run faster than against a view, at the cost of extra storage.
A streaming table is a Unity Catalog managed table that carries its own processing logic, expressed as flows that define it.
You can create materialized views and streaming tables in two ways: standalone from Databricks SQL, or inside Lakeflow pipelines. The docs also contrast their semantics directly: materialized views use batch semantics and streaming tables use streaming semantics.
| Object | Stores data? | How its contents are produced | Trade-off noted in the docs |
|---|---|---|---|
| Table | Yes; the default is a Unity Catalog managed table | Written directly with INSERT, UPDATE, DELETE, MERGE INTO | General-purpose; something else must load it |
| View | No; a virtual table | Its defining query runs when it is read | No duplicated storage; logic is encapsulated |
| Materialized view | Yes; precomputed query result | Defining query, refreshed via Databricks SQL or Lakeflow pipelines | Faster queries than a view, but extra storage |
| Streaming table | Yes; a Unity Catalog managed table | Flows that define and feed it | Uses streaming semantics; created standalone or in a pipeline |
Checkpoint 2 of 6· Exam question
A data engineering team is building a gold-layer object that BI analysts will hit with dozens of ad hoc dashboard queries throughout the day. The object aggregates daily revenue by region from cleaned silver-layer order data, and analysts want dashboard queries to return quickly without each query recomputing the aggregation from scratch, while the underlying numbers should stay reasonably current as new orders arrive. Which type of Unity Catalog object should the team build for this gold-layer aggregation?
Correct answer: A — Build the aggregation as a materialized view, so Databricks precomputes and caches the aggregated result and refreshes it as source data changes, letting dashboard queries read cached results instead of recomputing the join each time.
- A. This is correct. A materialized view precomputes and stores the query result, refreshing it as upstream data changes, so BI dashboard queries read cached results instead of recomputing the aggregation on every request.
- B. This is incorrect. Streaming tables process append-only source records exactly once and suit raw ingestion, not the role of serving a precomputed, repeatedly-queried aggregated summary for dashboards.
- C. This is incorrect. A standard view is not persisted and recomputes its defining query on every access, so dashboard queries would repeatedly re-run the full join and aggregation instead of hitting cached results.
- D. This is incorrect. Relying on a manually scheduled batch job to populate a plain table forces the team to build and maintain custom orchestration instead of using the declarative, automatically refreshed materialized view pattern.
Checkpoint 3 of 6· Match them up
Match each object to its defining characteristic
Tap a term, then the definition that fits it.
Only the view stores nothing. The table is changed by DML statements, the materialized view stores the result of its own query, and the streaming table carries its own flow logic.
“A view is a virtual table defined by a query that does not store data”Source: docs.databricks.com
Sources2
3.Views versus materialized views: compute on read or compute on refresh
You query views and materialized views the same way you query a table. The difference is when the work happens. A standard view recomputes its results on every query. A materialized view caches its results and refreshes them on a specified interval. Because the result is precomputed, queries against a materialized view can run much faster.
A materialized view is a declarative pipeline object with three parts: a defining query, a flow that updates it, and the cached results. It tracks changes in upstream data. When triggered, it incrementally processes what changed and keeps its output in sync with the sources. Databricks tries to process only what is needed. The result is always correct, even when that requires a full recomputation, but often only a few rows change. In the documented example, a sales_report grouped by country updates three rows instead of the whole report.
That is why the docs give "keeping a BI dashboard up to date with minimal end-user query latency" as a common use case.
Materialized views also have limits. Some input changes force a full recomputation, which can be expensive. Not all computations can be incrementally computed. Refreshes are not built for low latency, because an update takes seconds or minutes, not milliseconds. Unity Catalog stores the view's metadata. The cached data lives in cloud storage, and some backing data sits in the __databricks_internal catalog.
Checkpoint 4 of 6· Check yourself
A team needs a gold object that reflects source changes within milliseconds. Why is a materialized view a poor fit?
The docs say materialized views are not designed for low-latency use cases. They are queried like tables, are often refreshed incrementally, and Unity Catalog stores their metadata.
“The latency of updating a materialized view is in the seconds or minutes, not milliseconds.”Source: docs.databricks.com
4.Choosing the object for each gold dataset
The Lakeflow pipelines best-practices guide gives a decision table for pipeline dataset types. It covers streaming tables, materialized views and temporary views. Temporary views are pipeline-scoped and organize transformation logic without materializing data. Read with gold in mind, the table points to materialized views for aggregations, joins and dashboard acceleration, which are exactly the jobs the medallion page assigns to gold.
| Use case | Recommended type | Reason |
|---|---|---|
| Ingestion from cloud storage or a message bus | Streaming table | Processes each record once; handles high volume and append-only workloads. |
| CDC streams (inserts, updates, deletes) | Streaming table | Used as the target of AUTO CDC ... INTO for ordered, deduplicated CDC ingestion. |
| Complex aggregations and joins | Materialized view | Incrementally refreshed; avoids full recomputation on each update. |
| Dashboard query acceleration | Materialized view | Pre-computed results make queries faster than against raw tables. |
| Intermediate transformations (no downstream readers) | Temporary view | Organizes pipeline logic without incurring storage cost. |
Streaming tables are the right choice for ingestion and low-latency streaming transformations. They fit gold only when an analytics consumer really needs streaming, row-at-a-time results. Materialized views have one property that separates them from plain tables: you can't directly modify their data, because the query definition controls the output. If analysts or downstream jobs need to run UPDATE or MERGE INTO against a gold dataset, use a managed table. If the dataset is simply "this query, kept current," use a materialized view. If you want a reusable business definition that adds no storage and always runs against the latest data at read time, use a view.
The medallion page shows the canonical gold pattern: a weekly_bookings materialized view that preaggregates bookings and revenue, so analysts don't need to recreate frequently used aggregations themselves.
Checkpoint 5 of 6· Exam question
A junior data engineer proposes defining the gold-layer `customer_lifetime_value` object as a streaming table, reasoning that customers table updates frequently and a streaming table processes new records efficiently. The transformation logic recalculates lifetime value by aggregating a customer's full historical order history and applying corrections whenever late-arriving refund records are back-dated into earlier months. Why is a streaming table a poor fit for this specific gold-layer object?
Correct answer: A — A streaming table applies exactly-once, append-only processing to each record, so it cannot handle logic that must reprocess aggregated historical values or update earlier results because of back-dated refund corrections.
- A. This is correct. Streaming tables assume an append-only source and process each record exactly once, which does not accommodate aggregation logic that must revisit and correct historical results when back-dated refunds arrive.
- B. This is incorrect. Streaming tables are a core dataset type supported natively within Lakeflow Spark Declarative Pipelines; the mismatch here is about processing semantics, not about whether the object type is permitted.
- C. This is incorrect. Streaming tables are Unity Catalog managed tables registered in a normal catalog and schema, and BI tools can query them directly like any other table; internal backing storage details do not block that access.
- D. This is incorrect. Streaming tables can be defined with either SQL (`CREATE OR REFRESH STREAMING TABLE`) or Python; SQL support is available and does not require rewriting the logic in PySpark.
Checkpoint 6 of 6· Check yourself
A finance team wants a gold dataset of daily revenue by region that feeds a heavily used dashboard. Nobody needs to edit rows by hand. Which object does the guidance recommend?
Dashboard query acceleration and aggregations map to materialized views. Streaming tables are recommended for ingestion and CDC, and temporary views for intermediate steps with no downstream readers.
“Pre-computed results make queries faster than against raw tables.”Source: docs.databricks.com
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.A materialized view is the right gold object for millisecond-latency operational lookups because it is precomputed.Why is that wrong?
Precomputation speeds up reads, but refreshing the stored result takes seconds or minutes. Materialized views are not designed for low-latency use cases.
Covered in Views versus materialized views: compute on read or compute on refresh
2.You can fix a bad row in a gold materialized view with an UPDATE or MERGE INTO, as you would with a managed table.Why is that wrong?
A materialized view's contents are controlled only by its defining query. Direct DML is a property of tables, not materialized views.
Covered in Choosing the object for each gold dataset
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“The gold layer represents highly refined views of the data that drive downstream analytics, dashboards, ML, and applications.”
↩︎ What the gold layer is for“Optimizing gold-layer tables for performance is a best practice because these datasets are frequently queried.”
↩︎ What the gold layer is for“preaggregates this data so analysts and others don't need to recreate frequently used materialized views”
↩︎ Choosing the object for each gold dataset“Dimensional modeling and aggregation”
↩︎ Checkpoint - 2.
“The default table type created in Databricks is a Unity Catalog managed table.”
↩︎ Four objects, four storage and refresh behaviours“Materialized views use batch semantics and streaming tables use streaming semantics.”
↩︎ Four objects, four storage and refresh behaviours“A view is a virtual table defined by a query that does not store data”
↩︎ Checkpoint - 3.
“Because a materialized view is precomputed, queries against it can run much faster than against regular views.”
↩︎ Views versus materialized views: compute on read or compute on refresh“Unlike standard views, which recompute results on every query, materialized views cache the results and refresh them on a specified interval.”
↩︎ Key concept“They are not designed for low-latency use cases.”
↩︎ Exam trap 1“In this example, three rows update instead of the entire sales report.”
↩︎ Prediction“The latency of updating a materialized view is in the seconds or minutes, not milliseconds.”
↩︎ Checkpoint - 4.
“Keeping a BI dashboard up to date with minimal end-user query latency.”
↩︎ Views versus materialized views: compute on read or compute on refresh - 5.
“Streaming tables are the right choice for data ingestion and low-latency streaming transformations.”
↩︎ Choosing the object for each gold dataset“You can't directly modify the data in a materialized view; the query definition controls the output.”
↩︎ Exam trap 2“Pre-computed results make queries faster than against raw tables.”
↩︎ Checkpoint