Free Databricks Certified Data Engineer Associate Sample Questions

Correct answer: B — Databricks hosts the control plane in its own cloud account, while the data plane (compute and storage) resides in the customer's cloud account.

• A. Incorrect. In the standard Databricks architecture, the control plane is managed by Databricks in its own account, not within the customer's account.
• B. Correct. This describes the classic Databricks architecture. The control plane (residing in Databricks' cloud account) handles management tasks like UI, orchestration, and scheduling. The data plane (residing in the customer's cloud account) contains the compute clusters and the object storage where the data is actually stored and processed, ensuring the customer maintains physical control over their data.
• C. Incorrect. This is the reverse of the actual architecture; hosting the data plane in Databricks' account would violate the security requirement that data and compute reside in the customer account.
• D. Incorrect. Serverless compute actually moves the compute resources into the Databricks cloud account (the 'Serverless Data Plane'). This would contradict the institution's requirement that compute resources reside entirely within their own cloud provider account.
• E. Incorrect. Databricks never stores customer data in the control plane. The control plane only stores metadata and configuration (like notebooks and job definitions), while all customer data remains in the data plane.

Correct answer: C — Photon instances consume DBUs at a different, typically higher rate than standard instances, but overall costs may decrease if the job runs significantly faster.

• A. Incorrect. Photon is not free on Job clusters; it is a premium engine that incurs a higher DBU rate compared to standard non-Photon runtimes. While execution time may decrease, the total cost does not drop proportionally to the runtime reduction because the DBU rate is increased.
• B. Incorrect. Databricks billing remains based on DBUs, which measure compute consumption over time, not on a flat rate per gigabyte of data processed. Photon does not change the billing metric to a data-volume-based model.
• C. Correct. Photon instances consume DBUs at a higher rate than standard instances because Photon is a premium feature. However, the significant performance improvements can reduce total execution time, potentially lowering overall costs despite the higher per-hour rate.
• D. Incorrect. Photon is available on standard Job clusters, All-Purpose clusters, and SQL warehouses; it does not require switching to Serverless compute. Serverless compute has its own pricing model, but Photon can be enabled independently of that choice.

Correct answer: B — Store the credentials in Databricks Secrets and retrieve them using dbutils.secrets.get().

• A. Hardcoding credentials in notebooks is highly insecure and creates a risk of accidental exposure through source control, notebook sharing, or logs. Secure runtime isolation of clusters does not mitigate the risk of storing sensitive data in plain text code.
• B. Databricks Secrets is the recommended mechanism for storing sensitive credentials. Using dbutils.secrets.get() allows for programmatic retrieval at runtime without exposing secrets in notebook code, job definitions, or output logs, as Databricks automatically redacts secret values from output.
• C. Job parameters and widgets are intended for non-sensitive runtime inputs. Using them for credentials is insecure because plain text values can be exposed in the Databricks UI, job history, and configuration logs.
• D. Storing credentials in a CSV file, even within a Unity Catalog Volume, is not a secure credential management practice. Volumes are intended for file storage, and credentials stored this way lack the programmatic redaction and specialized access control policies provided by the Databricks Secrets utility.

Correct answer: A — FROM '/Volumes/main/sales/raw_data/'

• A. Correct. Unity Catalog Volumes are integrated into the Databricks File System (DBFS) and are accessible via the path format `/Volumes/<catalog>/<schema>/<volume>/`. When using `COPY INTO`, the source is specified as a string representing this path.
• B. Incorrect. `volume://` is not a valid URI scheme for referencing Unity Catalog Volumes in Databricks SQL or the `COPY INTO` command.
• C. Incorrect. `uc_volume()` is not a valid Databricks SQL function for defining data sources in a `COPY INTO` statement.
• D. Incorrect. The `COPY INTO` command requires a string literal representing a location in the `FROM` clause; `FROM VOLUME` is not a supported SQL syntax for this command.

Correct answer: A — from_json()

• A. from_json() is the correct function for converting a JSON string into a structured format (StructType or MapType). It requires a schema to be provided as the second argument to define the structure of the resulting column.
• B. to_json() is used to perform the inverse operation; it converts structured data (such as a struct, map, or array) into a JSON string representation.
• C. parse_json() is used to parse a JSON string into a 'Variant' type, which is Databricks' optimized format for semi-structured data. However, for converting specifically into a 'structured format' (like a named struct with a schema), from_json() is the standard choice.
• D. explode() is used to transform elements within an array or a map into separate rows. It does not parse JSON strings into structures.

Correct answer: A — Lakeflow Spark Declarative Pipelines using Auto Loader with expectations

• A. Correct. Lakeflow Spark Declarative Pipelines (part of Delta Live Tables) with Auto Loader are specifically designed for massive-scale file ingestion from cloud storage. Data quality 'expectations' allow you to define rules declaratively (e.g., ON VIOLATION DROP ROW) to enforce quality during the ingestion process before data lands in the target table. This fits the requirement for a managed pipeline with declarative DQ rules.
• B. Incorrect. While COPY INTO can load files and use a WHERE clause to filter data, it is a SQL command rather than a declarative, managed pipeline framework. It lacks the built-in expectation and monitoring framework provided by Lakeflow/DLT for sophisticated data quality enforcement.
• C. Incorrect. Lakeflow Connect with a JDBC driver is used for ingesting data from external relational databases, not for processing log files from cloud object storage.
• D. Incorrect. Databricks Partner Connect and webhooks are intended for third-party service integrations and event-driven notifications. They are not suitable for high-volume file ingestion or enforcing declarative row-level data quality rules.

Correct answer: A — option("cloudFiles.schemaEvolutionMode", "addNewColumns")

• A. Correct. According to Databricks documentation, the `addNewColumns` mode is designed for this scenario. When Auto Loader detects a new column, the stream will stop, but it updates the schema in its schema location. When the stream is restarted (often automatically by a Databricks Job), it will use the updated schema and ingest the new columns into the target Delta table.
• B. Incorrect. Setting `cloudFiles.inferSchema` to `false` would disable schema inference entirely. This would require you to provide a static schema manually and would prevent Auto Loader from automatically detecting and adding new columns.
• C. Incorrect. `mergeSchema` is an option for DataFrame write operations (e.g., `df.write.option("mergeSchema", "true")`), not an Auto Loader read option. Auto Loader uses the specific `cloudFiles.schemaEvolutionMode` option to manage schema changes at the source.
• D. Incorrect. This option does the opposite of what is required. It explicitly tells the stream to fail if a new column is detected and does *not* update the schema. This mode is used to enforce a strict, unchanging schema.

Correct answer: B — The Databricks workspace IP addresses have not been allowlisted in the MySQL database's network security group.

• A. Incorrect. The `ENABLE_CDC` property is related to Change Data Capture behavior and ingestion semantics for specific tables or schemas. It does not affect the initial network handshake or the ability to establish a connection to the source database, so it would not cause a 'Connection Timeout' error.
• B. Correct. A 'Connection Timeout' error indicates that Databricks compute resources cannot reach the source MySQL database over the network. In cloud environments, this is frequently caused by network security groups or firewalls that have not allowlisted the IP addresses of the Databricks workspace.
• C. Incorrect. The source database does not need to support Delta Lake; Delta Lake is the destination storage format used within Databricks. Lakeflow Connect handles the translation from the source database format to Delta Lake automatically, so this would not cause a connection timeout.
• D. Incorrect. The `USE WAREHOUSE` privilege is related to compute resources within Databricks. Lack of this privilege would result in an internal authorization or warehouse access error, not a network-level 'Connection Timeout' to an external MySQL source.

Correct answer: A — The stream stops with an `UnknownFieldException`, updates the schema in the schema location, and relies on the Databricks Workflow's retry mechanism to restart and process the new column.

• A. Correct. When `addNewColumns` is set, Auto Loader detects the new column, fails the stream with an `UnknownFieldException`, and automatically updates the schema in the `cloudFiles.schemaLocation` by appending the new column. Databricks recommends configuring the stream with Databricks Workflows (or Lakeflow Jobs) to automatically retry and restart, allowing the stream to resume with the updated schema. This is the documented behavior for `addNewColumns` mode.
• B. Incorrect. `addNewColumns` mode does not dynamically add columns without interruption; it requires a stream failure and restart. The stream stops with an `UnknownFieldException` before the schema is updated. While the new column is eventually added to the schema, the process is not seamless and requires a restart.
• C. Incorrect. This behavior describes the `none` mode, where new columns are silently ignored. In `addNewColumns` mode, the stream does not ignore new columns; it fails and updates the schema.
• D. Incorrect. Auto Loader does not drop files when encountering schema changes in `addNewColumns` mode. Instead, it fails the stream and updates the schema. The file is not skipped; it is processed after the stream restarts with the updated schema.
• E. Incorrect. Auto Loader does not have a built-in pause-and-alert mechanism for schema changes. In `addNewColumns` mode, the stream fails with an exception and relies on automated retries (e.g., via Databricks Workflows) to restart, not manual approval.

Correct answer: C — Set spark.sql.autoBroadcastJoinThreshold to 30MB

• A. Incorrect. This parameter controls the number of partitions used during shuffle operations (like joins or aggregations). Adjusting it can improve performance during a shuffle but it does not influence the Catalyst optimizer's decision to switch from a SortMergeJoin to a BroadcastHashJoin.
• B. Incorrect. Increasing the driver memory affects the memory available to the Spark driver process. While the driver is responsible for collecting and distributing broadcast variables, changing this memory limit does not trigger a change in the join strategy.
• C. Correct. The 'spark.sql.autoBroadcastJoinThreshold' parameter configures the maximum size, in bytes, for a table that will be broadcast to all worker nodes when performing a join. By increasing this threshold to 30MB (which is greater than the 25MB dimension table), Spark will automatically choose a BroadcastHashJoin, avoiding the shuffle of the 500GB table.
• D. Incorrect. This parameter determines the default number of partitions for RDD operations and may influence task parallelism. It does not control the high-level SQL join strategy or the threshold for broadcasting tables.

Correct answer: D — CONSTRAINT valid_sales EXPECT (sales_amount > 0)

• A. Incorrect. The `EXPECT OR DROP` syntax enforces the rule by dropping any rows that fail the condition from the target dataset. The requirement specifies that no records should be dropped.
• B. Incorrect. The `EXPECT OR FAIL` syntax causes the pipeline to stop and fail if any record violates the condition. This contradicts the requirement to allow the pipeline to continue running even if the rule is violated.
• C. Incorrect. While `CHECK` is a standard SQL constraint syntax used in many databases, Delta Live Tables (DLT) uses the specific `CONSTRAINT ... EXPECT` syntax to define data quality expectations that are monitored and displayed in the DLT UI and event logs.
• D. Correct. The `CONSTRAINT ... EXPECT` syntax is used to track data quality metrics (counting records that pass and fail) without modifying the dataset or stopping the pipeline. This allows all records to be processed while providing visibility into data quality issues.

Correct answer: A — january_sales.unionByName(february_sales, allowMissingColumns=True)

• A. Correct. `unionByName` aligns columns by name rather than by position. By setting the `allowMissingColumns` parameter to `True`, Spark handles DataFrames with different schemas by filling the missing columns in either DataFrame with nulls, allowing for a successful vertical combination.
• B. Incorrect. The `union` operation in PySpark is positional. It requires both DataFrames to have the exact same number of columns and compatible types in the same order. Since `february_sales` has an extra column, this would result in a schema mismatch error.
• C. Incorrect. A `join` (including an outer join) is a horizontal operation used to combine columns from different DataFrames based on a key. It is not used for vertical appending (stacking rows).
• D. Incorrect. `unionAll` is an alias for `union` in PySpark. It follows the same positional rules and requirements for identical schemas, meaning it would fail when encountering the additional column in `february_sales`.
• E. Incorrect. `crossJoin` performs a Cartesian product, which is a horizontal combination where every row of one DataFrame is paired with every row of the other. It is not a method for stacking DataFrames vertically.

Correct answer: B — Convert the Standard View to a Materialized View and schedule a weekly refresh.

• A. Global Temporary Views are session-scoped and do not persist data physically. They require the underlying query to be re-executed for each session, which would not reduce the 15-minute compute time or the associated costs.
• B. Materialized Views precompute and store the results of the query physically. Because the source data only changes weekly, scheduling a weekly refresh allows analysts to query the pre-calculated results instantly. This eliminates the need to run the expensive 15-minute aggregation repeatedly, significantly reducing compute costs.
• C. Streaming Tables are designed for incremental data ingestion and real-time processing. Using a streaming table with continuous processing for a dataset that only changes weekly would be unnecessarily complex and would likely increase compute costs rather than reduce them.
• D. Increasing the cluster size might reduce the execution time from 15 minutes, but it uses more expensive resources. This does not address the root problem of repeated execution of an expensive query, and it would likely result in higher overall compute costs.

Correct answer: D — sales_df.groupBy("store_id").agg(F.mean("amount"), F.count("transaction_id"))

• A. Incorrect. Applying the agg() function before groupBy() performs a global aggregation over the entire DataFrame first. Calling groupBy() after the data has already been aggregated into single values is logically incorrect for calculating per-store metrics.
• B. Incorrect. While groupBy("store_id").mean("amount") is valid for a single aggregate, you cannot chain another aggregate method like .count("transaction_id") directly onto the resulting DataFrame. To perform multiple different aggregations, the agg() method must be used.
• C. Incorrect. The select("store_id") call reduces the DataFrame to only the store column, removing the 'amount' and 'transaction_id' columns needed for the calculations. Additionally, mean() and count() are not DataFrame methods used in this specific chaining sequence.
• D. Correct. This is the standard PySpark pattern for multi-column aggregation. The groupBy("store_id") method creates a GroupedData object, and the agg() method allows the application of multiple aggregate functions (F.mean and F.count) simultaneously.
• E. Incorrect. PySpark does not support the aggregate(key="column") keyword argument syntax shown here. The correct method name is agg(), and it requires column expressions or a mapping dictionary.

Correct answer: A — logs_df.filter(col("error_code").isin(404, 500, 503))

• A. Correct. The `.isin()` method is the idiomatic PySpark way to filter a DataFrame based on a column matching any value in a set of provided literals. It translates to the SQL IN operator and is the most concise and efficient method for this requirement.
• B. Incorrect. This uses the Python `in` operator, which does not work with PySpark Column objects to generate a filter condition. Using it in this context will result in a TypeError or incorrect boolean evaluation rather than a Spark filter.
• C. Incorrect. The `.contains()` method is used for substring matching on string columns (e.g., checking if 'error' is in 'error_message'), not for checking membership in a list of numeric or discrete values.
• D. Incorrect. The equality operator `==` in PySpark expects a single scalar value or another column for comparison. It cannot be used to compare a column against a list of multiple possible values.
• E. Incorrect. The `array_contains()` function is designed to check if an array-type column contains a specific scalar value. In this scenario, `error_code` is a scalar column, and we are checking against multiple values, making this function inappropriate.

Correct answer: C — spark.read.table("bronze_sensors").selectExpr("CAST(reading_value AS DOUBLE) AS sensor_value")

• A. Incorrect. `expr()` is a function found in the `pyspark.sql.functions` module, not a method belonging to the DataFrame class. It cannot be chained directly after `spark.read.table()`.
• B. Incorrect. The `select()` method expects column objects or existing column names as strings. It does not interpret a string as a SQL expression with logic like CAST or AS; instead, it would look for a column literally named 'CAST(reading_value AS DOUBLE) AS sensor_value'.
• C. Correct. `selectExpr()` is a variant of `select()` that allows for SQL expressions as strings. It is specifically designed to interpret and execute SQL-style logic (like casting and aliasing) within a PySpark DataFrame transformation.
• D. Incorrect. The `withColumn()` method requires a column name (string) and a Column object as the second argument. It does not accept or evaluate raw SQL strings. To work, the second argument would need to be wrapped in the `expr()` function.
• E. Incorrect. The `sql()` method is a method of the `SparkSession` object (e.g., `spark.sql("...")`), not a method available on a DataFrame object returned by `spark.read.table()`.

Correct answer: B — The job automatically restarts using an exponential backoff mechanism.

• A. Incorrect. Continuous jobs are designed for resilience and high availability. Unlike standard jobs that might stay in a failed state, continuous jobs are architected to recover automatically without requiring manual intervention from an administrator.
• B. Correct. For a job with a 'Continuous' trigger, the default behavior in Databricks after a failure is to automatically restart. To manage resources efficiently and avoid rapid-fire failures during outages, the system utilizes an exponential backoff mechanism between restart attempts.
• C. Incorrect. Databricks does not automatically delete compute clusters as a response to a task failure. While notifications can be configured, the lifecycle of the cluster is not tied to the failure of the continuous task in this manner.
• D. Incorrect. Continuous jobs are not governed by cron schedules. They are designed for perpetual execution and do not have discrete 'run times' or interval ticks like cron-based scheduled jobs.

Correct answer: B — At Least One Succeeded

• A. The 'All Done' condition triggers the downstream task once all upstream dependencies have finished, regardless of their outcome (Success, Failed, Canceled, or Skipped). This does not meet the requirement of ensuring that at least one task succeeded before proceeding.
• B. The 'At Least One Succeeded' condition ensures that Task D will execute as long as at least one of the upstream tasks (Task A, Task B, or Task C) completes successfully. This matches the scenario where Task D should run even if the other two upstream tasks fail.
• C. The 'None Failed' condition requires that no upstream tasks result in a failure (they must be successful or skipped). Since the requirement explicitly states Task D should run even if two upstream tasks fail, this condition is incorrect.
• D. The 'All Succeeded' condition is the default behavior in Lakeflow Jobs and requires every upstream dependency to complete successfully. This is too restrictive for the stated goal of running after a single success.

Correct answer: A — A Databricks SQL warehouse

• A. Correct. A SQL Query task in a Databricks Job is designed specifically to execute saved SQL queries from the workspace. This task type requires a Databricks SQL warehouse (Serverless, Pro, or Classic) as the compute resource, as it utilizes the SQL execution environment rather than a standard Spark cluster.
• B. Incorrect. All-Purpose clusters are intended for interactive workloads and manual notebook execution. While they can run SQL within a notebook task, they cannot be used as the compute resource for a dedicated 'SQL Query' task type.
• C. Incorrect. Job clusters are the standard compute for automated tasks like Notebooks, Python scripts, or JARs. However, they are not compatible with 'SQL Query' tasks, which must run on a SQL warehouse.
• D. Incorrect. This compute type is associated with Delta Live Tables or Lakeflow declarative pipelines. It is not the correct resource for executing individual saved SQL queries from the workspace via the Jobs UI.
• E. Incorrect. A Single Node cluster is a specific configuration of an All-Purpose or Job Spark cluster used for small-scale Spark workloads and is not a SQL warehouse.

Correct answer: C — Click the "Pause" button on the job's schedule configuration in the Jobs UI.

• A. Incorrect. Deleting the job's schedule removes the complex cron expression entirely. This forces the engineer to manually recreate it after maintenance, which is inefficient and risks misconfiguration.
• B. Incorrect. Changing the trigger type from Scheduled to Manual discards the scheduled configuration. While it stops automatic runs, it does not preserve the schedule for easy resumption, making it more cumbersome than pausing.
• C. Correct. Clicking the "Pause" button in the Jobs UI temporarily halts scheduled executions while retaining the cron definition. The schedule can be resumed with a single click on Monday without re-entering any configuration.
• D. Incorrect. Setting the maximum worker count to 0 affects compute resources, not the job trigger. The job would still attempt to run on schedule but would fail or hang due to lack of resources, generating unnecessary alerts.

Correct answer: C — Replace the Scheduled trigger with a File arrival trigger.

• A. Incorrect. Running the job weekly would only delay processing further, causing logs to accumulate for up to a week before being processed. This does not address the requirement to process logs as soon as they arrive.
• B. Incorrect. While a larger cluster may speed up processing when the job eventually runs, it does not trigger the job to start earlier. The job would still only run at midnight, so late-arriving logs would still not be processed until the next scheduled run.
• C. Correct. According to Databricks documentation, File arrival triggers are recommended for scenarios where new data files land in storage at irregular intervals. A File arrival trigger monitors a specified cloud storage path and automatically initiates the job when new files are detected, typically checking every minute. This ensures that data processing begins as soon as data is available, eliminating the latency caused by fixed schedules.
• D. Incorrect. This still uses a fixed scheduled trigger, just one minute earlier. Logs arriving at 2:00 AM would still be missed by that day's run and only processed the following day.
• E. Incorrect. Continuous execution is a mode for Structured Streaming queries that keeps the cluster running and processes data continuously as it arrives. The existing job is a batch job, not a streaming job. While converting to a streaming job could be an alternative, the question asks for a change to the existing job. A File arrival trigger for a batch job is the more straightforward and recommended solution for this pattern of irregular file arrival.

Correct answer: B — Define the `raw_data_path` variable under the `variables` mapping within each specific `target` block in the `databricks.yml` file.

• A. Creating separate `databricks.yml` files is not the intended pattern for Databricks Asset Bundles. Bundles are designed to use a single declarative file with multiple 'targets' (e.g., dev, prod) to keep deployment logic consistent and maintainable.
• B. Defining environment-specific values under the `variables` mapping within each `target` block is the standard approach in DABs. This allows the bundle to resolve the `raw_data_path` variable differently depending on which target is being deployed (e.g., `databricks bundle deploy --target prod`).
• C. Hardcoding environment logic in pipeline source code is an anti-pattern. It makes the code less portable and harder to manage. DABs are intended to externalize environment differences into the configuration layer rather than the application logic.
• D. While variables can be overridden via CLI, this is not the recommended way to manage standard environment configurations. Storing these values in the `databricks.yml` file under specific targets ensures that the environment definitions are version-controlled and reproducible.
• E. Databricks Secret Scopes are reserved for sensitive credentials (like API keys or passwords). Non-sensitive information such as directory paths should be handled via bundle configuration variables to ensure clarity and proper metadata management.

Correct answer: D — databricks bundle init

• A. Incorrect. There is no Databricks CLI command called 'databricks bundle create' for creating a new bundle project from a template. Valid lifecycle commands include init, validate, deploy, and run.
• B. Incorrect. 'databricks bundle new' is not a valid command in the Databricks CLI for bundle project creation.
• C. Incorrect. 'databricks bundle generate' is not the standard command to create a new bundle project. The CLI uses 'init' to scaffold the project structure.
• D. Correct. 'databricks bundle init' is the standard command used to initialize a new Databricks Asset Bundle (DAB) project. It scaffolds the necessary directory structure and configuration files, often by prompting the user for a template.
• E. Incorrect. While 'scaffold' describes the action the command performs, 'databricks bundle scaffold' is not a valid command in the Databricks CLI.

Correct answer: A — complex

• A. Correct. In Databricks Asset Bundles (DABs), the `complex` type is the specific keyword used in the `variables` block to define variables that hold structured data, such as objects (maps of key-value pairs) or arrays.
• B. Incorrect. While 'object' is a common term for structured data in many programming contexts, it is not a valid keyword for the `type` property in a Databricks Asset Bundle variable definition. The correct keyword is `complex`.
• C. Incorrect. Although a map of key-value pairs is conceptually a 'map', this is not a recognized type identifier in the DABs configuration schema. All non-scalar structures are categorized under the `complex` type.
• D. Incorrect. `dict` is a term commonly used in Python to describe dictionaries, but it is not a valid configuration type for Automation Bundle variables.

Correct answer: B — The `mode: development` setting automatically prefixes resource names with the developer's username and pauses schedules to prevent accidental runs.

• A. Incorrect. The `--release` flag is not a valid deployment flag for Databricks Asset Bundles and has no effect on naming prefixes or schedule states. These behaviors are controlled solely by the deployment mode specified in the bundle configuration.
• B. Correct. When `mode: development` is set, Databricks Asset Bundles automatically prefix resource names with the developer's username to prevent naming collisions and pause all schedules to avoid unintended job executions and costs in a development environment.
• C. Incorrect. The prefix and paused schedules are intentional features of development mode designed for safe testing, not a fallback triggered by a CLI error. The Databricks CLI does not enter a safe mode on failure.
• D. Incorrect. While workspace administrators can set policies, the specific behavior of adding a username prefix and pausing schedules is a built-in feature of Databricks Asset Bundles' development mode, not an enforced policy.
• E. Incorrect. The behavior is caused by the explicit `mode: development` setting, not by the absence of a `production: true` flag. Databricks Asset Bundles do not use a `production: true` flag; instead, you would set `mode: production` to disable these development behaviors.

Correct answer: A — By clicking the "Create Pull Request" link in the Git dialog, which opens the Git provider's UI to complete the pull request.

• A. Correct. After pushing changes to a feature branch, the Git dialog in Databricks displays a "Create Pull Request" link. Clicking it opens the connected Git provider's UI, where the engineer can complete the pull request and initiate the code review process.
• B. Incorrect. Selecting the `main` branch and clicking "Merge into current" performs a local merge within the Databricks repository. This does not create a pull request or trigger a remote code review on the Git provider's platform.
• C. Incorrect. The `dbutils` library does not contain a `git` module or a `createPR()` command. Pull requests must be created through the Databricks UI or the Git provider's interface, not via notebook commands.
• D. Incorrect. The "Commit & Push" dialog is used to push changes to the remote repository, but it does not include a "Create PR" checkbox. Pull requests are initiated separately after pushing, using the Git dialog's dedicated link.
• E. Incorrect. Workspace settings are for administrative configurations and do not include an "Initiate Code Review" option. Pull requests are managed through the Git dialog or the external Git hosting service, not workspace settings.

Correct answer: B — Enable Adaptive Query Execution (AQE) skew join optimization or manually salt the join keys

• A. Incorrect. Increasing driver memory does not address an Executor OOM caused by a skewed task. The driver coordinates tasks and manages the plan, while executors handle the actual data processing. The issue here is that one partition is disproportionately large, causing a specific executor to fail, not the driver.
• B. Correct. A single task processing 99% of the data is a classic sign of data skew, which causes one executor to be overwhelmed and run out of memory. Enabling Adaptive Query Execution (AQE) skew join optimization allows Spark to dynamically handle skewed partitions by splitting them into smaller chunks. Alternatively, manually salting the join keys redistributes the skewed data more evenly across the cluster.
• C. Incorrect. Calling df.collect() on a 50 GB dimension table is inappropriate as it pulls all that data into the driver's memory, likely leading to a Driver OOM error. Furthermore, this action does not address the data skew occurring during the join operation.
• D. Incorrect. Caching a 10 TB fact table is impractical for almost any cluster and would likely worsen memory pressure. Caching is used for performance when reusing data, but it does not resolve the root cause of the failure, which is the uneven distribution of data (skew) during the join.

Correct answer: A — ARRAY

• A. Correct. ARRAY is a complex (collection) data type and is not supported as a clustering key in Liquid Clustering. Clustering columns must be simple, orderable primitive types that can be used for efficient range-based layout decisions and data skipping.
• B. Incorrect. STRING is a primitive data type supported as a clustering key. It is an orderable type often used to define clustering keys for optimizing data layout on high-cardinality columns.
• C. Incorrect. TIMESTAMP is a supported primitive data type for Liquid Clustering. It is a common choice for clustering when queries frequently filter by time ranges or event dates.
• D. Incorrect. INT is a supported numeric primitive data type. Numeric types are valid clustering columns because they are orderable and work effectively for data skipping and layout optimization.

Correct answer: A — Use the "Launch type" filter dropdown to display only "Scheduled" runs.

• A. Correct. The "Launch type" filter in the Databricks Lakeflow Jobs run history view allows users to narrow down the displayed executions by how they were triggered (e.g., Scheduled, Manual, or API). Selecting "Scheduled" will filter out manually triggered test runs, isolating the nightly production runs and providing a clear, consistent historical baseline in the duration chart.
• B. Incorrect. While grouping by job parameters can provide insights into specific configurations, it is not the intended mechanism to isolate scheduled versus manual runs. Furthermore, this would only be effective if such a parameter were consistently and correctly populated for every run, which is less reliable than using the built-in launch type metadata.
• C. Incorrect. Sorting the list by duration changes the order of the table entries but does not remove data points from the duration chart. The chart would still plot all runs, including the manual test payloads, leaving the erratic visual trend unresolved.
• D. Incorrect. Switching between "List" and "Matrix" modes changes the visualization format of the job runs (Matrix mode is typically used to visualize the status of tasks across multiple runs). It does not filter the dataset used for the duration trend chart.

Correct answer: A — The cluster is spending more time assigning and coordinating tasks than actually processing data, often due to an excessive number of tiny tasks.

• A. Correct. Long 'Scheduler Delay' bars and short 'Executor Computing Time' bars indicate that the cluster is spending more time assigning and coordinating tasks than actually processing data. This pattern is a classic sign of an excessive number of tiny tasks, causing the Spark driver to be overwhelmed with scheduling overhead.
• B. Incorrect. Disk spilling during a wide transformation would appear as increased execution time and is tracked via spill metrics or elevated shuffle write/read times, not as uniformly long 'Scheduler Delay' bars. The described pattern points to task scheduling overhead, not I/O delays from spilling.
• C. Incorrect. Data skew is characterized by high variance in task execution times, where a few tasks have very long green 'Executor Computing Time' bars while others finish quickly. It does not produce uniformly long 'Scheduler Delay' bars across almost every task.
• D. Incorrect. Driver memory pressure can cause job slowdowns or failures, but it does not manifest as long 'Scheduler Delay' bars for every task in the stage timeline. This visual pattern specifically indicates task scheduling overhead, not driver-side memory issues.
• E. Incorrect. Network I/O delays for fetching shuffle blocks are captured as 'Shuffle Read Fetch Wait Time' within the task execution period, not as part of the orange 'Scheduler Delay' bar. The observed pattern of short compute time and long scheduler delay points to excessive task coordination, not shuffle fetch waits.

Correct answer: B — The task node shows a "Succeeded" status and includes an "Attempts" counter in its run details.

• A. Incorrect. In the Lakeflow Jobs UI, yellow is typically used to indicate a 'Running' or 'Waiting' state, not a warning for a task that succeeded after retries. A task that ultimately succeeds does not display a yellow warning state or a transient failure flag.
• B. Correct. When a task fails initially but succeeds on a retry, its final status is shown as 'Succeeded' (green) because the task ultimately completed successfully. The run details panel includes an 'Attempts' counter that records the number of tries, including the initial failure and subsequent retries.
• C. Incorrect. The DAG task graph represents the logical flow of the job and does not dynamically split into separate nodes for each retry attempt. Each task instance remains a single node in the visualization, regardless of retries.
• D. Incorrect. A 'Skipped' status indicates that a task was not executed, usually due to an upstream dependency failure or a conditional run state. The UI does not handle retries by marking the original task as 'Skipped' or by cloning it into new nodes.

Correct answer: B — Service Principal

• A. A workspace admin is an administrative role assigned to human users or groups. It is tied to a human user's identity and specific permissions within a workspace, making it unsuitable and insecure as a principal for automated, non-human production jobs.
• B. A service principal is a non-human identity created in Databricks for use with automated tools, jobs, and applications. It is the recommended best practice for production workloads because it is not tied to an individual user's account, ensuring that jobs remain stable and auditable even if a human user leaves the organization.
• C. An instance profile is a cloud-specific mechanism (primarily AWS) used to associate IAM roles with compute resources. It is not a principal type within the Databricks Unity Catalog identity hierarchy for managing access via SQL privileges.
• D. A Personal Access Token (PAT) is an authentication credential, not a principal type itself. PATs are typically tied to a specific human user's identity and lifecycle; using them for automation is discouraged in favor of service principals.

Correct answer: A — ALTER TABLE employees ALTER COLUMN ssn SET MASK ssn_mask_fn;

• A. Correct. In Unity Catalog, masking is applied to an existing column using the syntax `ALTER TABLE <table_name> ALTER COLUMN <column_name> SET MASK <mask_function>`. This assigns the masking function to the column so that query results are dynamically transformed based on the user's permissions.
• B. Incorrect. The `ADD MASK ... ON` syntax is not the correct Databricks SQL syntax for applying a column mask to an existing table. The correct approach requires the `ALTER COLUMN ... SET MASK` clause.
• C. Incorrect. This command would physically overwrite the original data in the table with masked values. Column masking in Unity Catalog is designed to change how data is presented at query time without altering the underlying storage.
• D. Incorrect. There is no SQL syntax to `GRANT MASK` on a specific column. Column-level security is managed by defining a masking function (which contains the conditional logic) and then applying that function to the column via `ALTER TABLE`.

Correct answer: A — DESCRIBE EXTENDED patient_records;

• A. Correct. The `DESCRIBE EXTENDED` command (or its synonym `DESCRIBE TABLE EXTENDED`) provides detailed metadata about the table. In Unity Catalog, this command is used to inspect table properties, including applied row filters and column masking functions. Row filters typically appear in the table properties section, while masking functions are listed alongside the specific column definitions.
• B. Incorrect. `SHOW FILTERS AND MASKS ON` is not a valid Databricks SQL command. While the name sounds intuitive, it does not exist in the Databricks SQL syntax for Unity Catalog governance.
• C. Incorrect. The `DESCRIBE GRANTS ON TABLE` command is used to display the privileges and permissions (such as SELECT, MODIFY, or OWN) that have been granted to principals (users or groups) on the table. It does not provide information about row-level or column-level security policies.
• D. Incorrect. `SHOW ACCESS POLICIES FOR` is not a standard Unity Catalog command. To inspect data governance policies like filters and masks, engineers should use table description commands or query the Information Schema.

Correct answer: A — `CREATE EXTERNAL TABLE`

• A. Correct. In Unity Catalog, to create an external table, the user must have the `CREATE EXTERNAL TABLE` privilege on the External Location that points to the storage path. This is a specific privilege designed to authorize metadata registration for external data.
• B. Incorrect. `CREATE MANAGED TABLE` is not a standard privilege for External Locations. Creating managed tables typically requires permissions on the schema or catalog, and the location is managed automatically by the metastore or via `CREATE MANAGED STORAGE`.
• C. Incorrect. `WRITE FILES` is a privilege on an External Location that allows users to write data files directly using Spark or the `COPY INTO` command, but it does not grant the specific permission required to create an external table object in the metastore.
• D. Incorrect. `MODIFY` is a privilege granted on tables or views to allow users to perform DML operations (UPDATE, DELETE, MERGE). It is not used as a privilege for External Location objects.
• E. Incorrect. `USE LOCATION` is not a valid privilege in the Unity Catalog security model. The correct administrative privilege to utilize a storage path for an external table is `CREATE EXTERNAL TABLE`.