Free Microsoft Certified: Fabric Data Engineer Associate (DP-700) Sample Questions

Correct answer: B — The Workspace Runtime version in Spark Settings.

• A. Incorrect. The Default Lakehouse setting is used to specify which lakehouse is automatically attached to new notebooks and SQL endpoints as the default storage surface. It does not control the compute runtime version.
• B. Correct. Microsoft Fabric bundles specific versions of Spark and Delta Lake into Runtimes. The Workspace Runtime version, found in the Spark Settings of a workspace, allows an administrator to set the default runtime that all new notebooks and Spark job definitions will use. Setting this to a runtime that includes Spark 3.4 and Delta Lake 2.4 achieves the desired outcome.
• C. Incorrect. The High Concurrency Mode toggle optimizes Spark session management for scenarios with many concurrent users or queries. It affects how Spark resources are shared and reused but does not determine the version of Spark or Delta Lake being used.
• D. Incorrect. Environment variables are typically used to pass configuration parameters or secrets to a Spark application. While they can influence runtime behavior, they do not set the default, base runtime version for new items in a workspace. This setting is also configured in the Admin Portal at the tenant level, whereas the question concerns a specific workspace.

Correct answer: B — Public Libraries > Add from PyPI

• A. Incorrect. The 'Spark Compute > Spark Properties' section is used to control Spark runtime configurations, such as driver/executor memory settings and other Spark-specific configuration keys. It is not designed for managing or pinning specific versions of Python libraries within the Environment.
• B. Correct. This is the correct location. The 'Public Libraries > Add from PyPI' interface is specifically designed to add Python packages from the public Python Package Index (PyPI). Here, you can search for a library like `pandas` and explicitly specify the required version (e.g., `pandas==1.5.0`) to ensure that all notebooks and jobs using this Environment use that exact version.
• C. Incorrect. The 'Resources > Upload' section is intended for uploading custom files, such as custom-built Python wheel (`.whl`) files or other necessary resources. While you could technically upload a specific wheel file for a library, the standard and more direct method for managing public libraries from PyPI is through the 'Public Libraries' section.
• D. Incorrect. 'Data Engineering > General' refers to a high-level settings area within the Data Engineering workload or workspace. It does not contain settings for managing the specific library dependencies of a custom Environment.

Correct answer: A — ALTER TABLE Orders ALTER COLUMN CreditCard ADD MASKED WITH (FUNCTION = 'partial(0, "XXXX-XXXX-XXXX-", 4)')

• A. This is the correct statement. It uses the `ALTER TABLE...ADD MASKED WITH` syntax to apply Dynamic Data Masking. The `partial()` function is used to specify a custom mask, showing a prefix of 0 characters, a padding string of 'XXXX-XXXX-XXXX-', and a suffix of the last 4 characters, which precisely meets the requirement.
• B. This is incorrect. The `default()` function applies a full mask based on the column's data type. For a string column like `CreditCard`, it would typically mask the entire value (e.g., as 'xxxx'), not just a portion of it.
• C. This is incorrect. `GRANT UNMASK` is a permission statement that allows a specific user or role to view the unmasked data. It does not define or apply the mask itself. Furthermore, the `UNMASK` permission is granted to principals (like users or roles), not directly to a table column.
• D. This is incorrect for two reasons. First, creating a view is a different method of securing data and is not Dynamic Data Masking. Second, the `LEFT(CreditCard, 4)` function would expose the *first* four digits of the credit card, which is the opposite of the requirement to show the *last* four digits.

Correct answers: A, B, C — A Security Policy; A Filter Predicate Function (TVF); A User or Role

• A. Correct. A security policy is the core component that enforces row-level security (RLS). It binds the filter predicate function to the target table, ensuring that the defined access logic is automatically applied whenever the table is queried.
• B. Correct. A filter predicate function, typically implemented as an inline table-valued function (TVF), contains the specific logic for filtering the rows. This function evaluates each row and determines whether the current user should have access, for example, by comparing the value in the 'Department' column to the user's identity or role membership.
• C. Correct. The entire RLS mechanism is based on the context of the user or role executing the query. The logic within the filter predicate function evaluates against the current user's identity (e.g., using `USER_NAME()` or `SESSION_CONTEXT()`) or their role membership to determine which rows are visible. Permissions to access the table are granted to users or roles, and the policy then filters the data for them.
• D. Incorrect. A data masking rule is used to obfuscate or hide specific data within a column for non-privileged users (e.g., showing 'xxx-xx-1234' instead of a full social security number). It does not filter or remove entire rows from a result set.
• E. Incorrect. A sensitivity label is a feature of Microsoft Purview used for data classification. It helps categorize data based on its sensitivity (e.g., Confidential, Public) to aid in governance and protection policies, but it does not directly implement row-level filtering logic.

Correct answers: A, B — Data source connection string; Report parameters

• A. Correct. In Fabric deployment pipelines, you can create deployment rules to parameterize data source connection strings. This is a critical feature for lifecycle management, allowing a single paginated report to connect to different data sources across development, test, and production environments without modifying the report definition itself.
• B. Correct. Deployment rules for paginated reports also support parameterizing the report's parameters. This enables you to set different default values or other properties for parameters depending on the target stage, which enhances the report's flexibility and reusability in various environments.
• C. Incorrect. The visual layout is an integral part of the report's design, defined within the Report Definition Language (RDL) file. Deployment rules are designed to configure properties and connections, not to alter the fundamental design or visual structure of the report.
• D. Incorrect. The report name is the identifier of the artifact within the workspace. Deployment rules act upon an existing report to change its configurable properties; they do not support renaming the report as part of the deployment process.

Correct answer: A — The pipeline in Test points to 'LH_Dev'.

• A. This is correct. By default, Microsoft Fabric deployment pipelines copy artifacts and their configurations exactly as they are from the source to the target workspace. Without deployment rules or parameterization configured, the pipeline's reference to the 'LH_Dev' Lakehouse is copied as-is. Therefore, the newly deployed pipeline in the Test environment will still point to the original development Lakehouse.
• B. Incorrect. The deployment process does not automatically detect and remap connections based on environment names or conventions. To achieve this behavior, you must explicitly configure a deployment rule to change the data source from 'LH_Dev' to 'LH_Test' when deploying to the Test stage.
• C. Incorrect. The pipeline is not broken; it has a valid, configured connection. The issue is that it's connected to the wrong resource for the Test environment. The pipeline would likely fail at runtime if the Test environment's service principal or user identity doesn't have access to 'LH_Dev', but immediately after deployment, the connection configuration itself is intact.
• D. Incorrect. The deployment process will succeed because its primary function is to copy the artifacts. It does not validate the logical correctness of connection strings for the target environment. A deployment failure would typically be caused by issues like missing dependent items, insufficient permissions, or validation errors during the copy process, not by an unconfigured connection.

Correct answer: B — Dataflow Gen2

• A. Incorrect. Notebooks in Microsoft Fabric are code-centric environments for languages like PySpark, Spark SQL, and Scala. While they are powerful for complex transformations and can write to a Lakehouse, they do not meet the requirement for a low-code, visual interface.
• B. Correct. Dataflow Gen2 is the Power Query experience within Microsoft Fabric. It provides a rich, low-code/no-code visual interface for data ingestion and transformation. It allows users to perform column mapping, data type conversion, and apply complex conditional logic through a graphical user interface, and it is designed to natively output data to Fabric destinations like the Lakehouse.
• C. Incorrect. The Copy Data activity in a Fabric Pipeline is primarily designed for high-performance data movement. While it supports basic schema mapping and data type conversion, it lacks the advanced, visual transformation capabilities and conditional logic functionalities that are required. It is not a data wrangling tool.
• D. Incorrect. A Spark Job Definition is used to submit and run pre-compiled Spark jobs (e.g., Jar files). This requires writing and packaging Spark code beforehand and is not a low-code, visual authoring tool for data transformation.

Correct answer: B — On-premises data gateway

• A. Incorrect. Azure ExpressRoute provides a private, dedicated network connection between an on-premises network and Azure. While it can be used for the underlying network transport, it is not the specific component that Dataflow Gen2 uses to connect to the data source. The On-premises data gateway is still required to act as the bridge between the cloud service and the on-premises database.
• B. Correct. The On-premises data gateway is designed to act as a secure bridge, providing data transfer between on-premises data sources (like a SQL database behind a firewall) and Microsoft cloud services such as Microsoft Fabric, Power BI, and Logic Apps. It is installed within the local network and manages the communication, allowing Dataflow Gen2 to securely query the on-premises database without exposing it to the internet.
• C. Incorrect. Trusted Workspace Access is a feature related to services like Azure Synapse Analytics, which allows other trusted Azure services to securely access data within the workspace. It is not a component for establishing connectivity to on-premises data sources.
• D. Incorrect. A Virtual Network (VNet) data gateway is used to connect Microsoft cloud services to data sources that are secured within an Azure Virtual Network. It is for Azure-native resources, not for resources located in an on-premises network. For on-premises connectivity, the On-premises data gateway is the correct component.

Correct answer: A — True

• A. The statement is correct. Microsoft Fabric provides two primary ways to assign a workspace to a domain. One method is centrally through the Admin portal, and the other is directly from the settings of the individual workspace. A Fabric administrator or a workspace administrator can navigate to the workspace's settings, find the 'Domain' property, and select the appropriate domain from the list. This action successfully assigns the workspace and is a key feature for delegating management.
• B. The statement is not false. The procedure described—assigning a workspace to a domain via its own settings—is a valid and supported method in Microsoft Fabric. While it's also possible to perform this assignment from the Admin portal, the availability of the option within the workspace settings makes the original statement true.

Correct answer: B — `email()`

• A. Incorrect. The `default()` function provides a full mask based on the data type. For string data types, it replaces the entire value with 'XXXX', which does not meet the requirement to show the first letter of the email address.
• B. Correct. According to Microsoft documentation for Dynamic Data Masking in Fabric Data Warehouse, the `email()` function is the specific built-in function for masking email addresses. It exposes the first letter and appends a constant string, resulting in a format like `aXXX@XXXX.com`.
• C. Incorrect. The `partial()` function applies a custom mask. The parameters `(1, 'XXXX', 0)` would expose the first character and then mask the rest of the string with 'XXXX'. This would not produce the required email-like format (e.g., `aXXX@XXXX.com`).
• D. Incorrect. The `random()` masking function is used only for numeric data types to mask the original value with a random value within a specified range. It cannot be applied to a string column like `Email`.

Correct answer: B — Member

• A. Incorrect. While the `Admin` role can update the app and share items, it also grants the ability to add or remove other users, including other Admins. This violates the requirement that UserA should not be able to add other Admins.
• B. Correct. The `Member` role in a Microsoft Fabric workspace allows a user to update the workspace app and share items. Crucially, a Member can add other users to the workspace but cannot add or remove users with the `Admin` role, perfectly matching the principle of least privilege for this scenario.
• C. Incorrect. The `Contributor` role in a Microsoft Fabric workspace allows a user to create, edit, and delete content within the workspace, but it does not grant permission to publish or update the workspace app. Therefore, this role does not meet all the requirements.
• D. Incorrect. The `Viewer` role provides read-only access to the workspace. A user with this role can view and interact with reports and other items but cannot update the app or share items, making it insufficient for the specified tasks.

Correct answer: B — Connect the Notebook activity to the Email activity with a 'Failure' dependency path (red arrow), and to the Dataflow activity with a 'Success' dependency path (green arrow).

• A. Incorrect. The 'Skipped' dependency path is triggered when an upstream activity is not executed because its preceding conditions were not met, not when it fails. The requirement is to send an email on failure, so a 'Skipped' path does not address the failure scenario.
• B. Correct. Using a 'Failure' dependency path (red arrow) from the Notebook activity to the Email activity ensures the email is sent only if the notebook fails. A 'Success' dependency path (green arrow) to the Dataflow activity ensures it runs only on success, directly implementing the required orchestration.
• C. Incorrect. A Switch activity is designed to branch based on evaluating an expression against discrete values, which is overly complex for simple success/failure branching. The built-in dependency paths are the standard and intended pattern for this scenario.
• D. Incorrect. A ForEach loop is meant for iterating over collections, not for a single notebook execution. While a 'Completion' dependency exists, it runs regardless of outcome and does not allow separate actions for success and failure.

Correct answer: D — Set the Automatic Pause limit.

• A. Incorrect. Executor Memory limits RAM per task determines the memory allocated to each executor for task execution. This setting affects job performance and memory usage but does not control how long an idle Spark session remains active before termination.
• B. Incorrect. Node Family selects lower-cost VMs, which influences the type and cost of virtual machines used for Spark nodes. While this can reduce overall compute costs, it does not manage the automatic termination of idle sessions.
• C. Incorrect. Autotune adjusts job configurations automatically to optimize performance, such as tuning executor counts or memory settings. Although it can improve resource efficiency, it does not set an idle timeout for pausing or terminating sessions.
• D. Correct. Set the Automatic Pause limit defines the idle timeout after which an inactive Spark session is automatically paused and its resources released. Lowering this value ensures idle sessions are terminated quickly, directly reducing costs.

Correct answer: A — Azure Data Studio with the SQL Database Projects extension.

• A. Correct. Azure Data Studio with the SQL Database Projects extension is an officially recommended approach for implementing definitions and procedures in Azure Synapse Analytics (dedicated and serverless SQL pools) and is compatible with Fabric Data Warehouse. It supports database-as-code, source control integration, and CI/CD pipelines through .dacpac deployment. This method is documented by Microsoft for managing SQL objects like tables and stored procedures.
• B. Incorrect. Power BI Desktop is a data visualization and reporting tool, not designed for creating or publishing SQL Database Projects. Power Query M is used for data transformation within Power BI, not for defining or deploying DDL scripts to a data warehouse.
• C. Incorrect. Fabric Pipeline Copy Activity is used for data movement and orchestration, not for creating SQL Database Projects. While it can invoke stored procedures, it does not provide source control or project-based management of DDL definitions.
• D. Incorrect. Microsoft Excel with Power Query is a data analysis tool that can connect to data sources, but it cannot create or publish SQL Database Projects. It lacks the capabilities for database schema management, source control, or deployment to a Fabric Data Warehouse.

Correct answer: B — A Shortcut to Amazon S3

• A. Incorrect. A Copy Data activity pipeline is designed specifically to move or copy data from a source to a destination. This action would directly violate the corporate policy that prohibits duplicating or physically moving the data into the Fabric capacity due to data sovereignty rules.
• B. Correct. A Shortcut in Microsoft Fabric is a symbolic link or virtual reference to data stored in an external location, such as Amazon S3. It allows Fabric engines, including Spark, to access and query the data in-place without physically copying or moving it into OneLake storage. This approach directly addresses the requirement to honor data sovereignty rules by keeping the data in its original location.
• C. Incorrect. A Dataflow Gen2 query is used for data transformation and ingestion. While it can read from S3, its primary purpose is to process and load the data into a destination within Fabric, such as a Lakehouse table. This process would inherently create a copy of the data, violating the stated policy.
• D. Incorrect. The Mirrored Database feature is designed to replicate data from operational databases (like Azure SQL DB or Snowflake) into Fabric. This involves creating a read-only copy of the data, which explicitly violates the policy against data duplication. Furthermore, this feature is not applicable for accessing file-based data in Amazon S3.

Correct answer: B — spark.microsoft.delta.optimizeWrite.enabled

• A. Incorrect. The `spark.sql.parquet.compression.codec` configuration controls the compression algorithm (e.g., snappy, gzip) used for Parquet files. While this affects storage size and I/O performance, it is not the specific configuration that optimizes the file layout and size for efficient reads by Power BI Direct Lake.
• B. Correct. The `spark.microsoft.delta.optimizeWrite.enabled` configuration is a crucial Spark setting within Microsoft Fabric and Azure Synapse Analytics. When enabled (set to `true`), it dynamically optimizes the file sizes as data is being written. This reduces the number of small files, a common performance bottleneck, and creates a data layout that is highly beneficial for the read performance of analytics engines, including Power BI in Direct Lake mode.
• C. Incorrect. V-Order is a proprietary write-time optimization technique in Microsoft Fabric that applies a specific sorting and layout to Parquet files to improve compression and read performance. However, it is an underlying engine feature and optimization process, not a Spark configuration that is enabled with a flag named 'V-Order' in a Notebook or job.
• D. Incorrect. Auto-compaction is a feature of Delta Lake that automatically merges small files into larger, more optimal ones. It is enabled via a Delta table property (`delta.autoOptimize.autoCompact = true`), not a Spark session configuration like the one requested in the question.

Correct answer: B — The SQL Analytics Endpoint of the Lakehouse

• A. Incorrect. The Spark Job Definition endpoint is used for submitting and managing Spark jobs, not for direct, ad-hoc SQL querying of files. While Spark SQL can be used within a Spark job, this endpoint itself is not the interactive query surface needed.
• B. Correct. The SQL Analytics Endpoint of a Lakehouse is specifically designed to provide a serverless T-SQL query surface over all data within that Lakehouse, including files accessed via shortcuts. It allows users to run SQL queries directly on files like CSVs without needing to explicitly load them into a managed Delta table first.
• C. Incorrect. The Dataflow Gen2 endpoint is used for data ingestion and transformation using a Power Query experience. It is an ETL/ELT tool, not a direct SQL query endpoint for ad-hoc analysis on files in the lake.
• D. Incorrect. The XMLA (XML for Analysis) endpoint is used for connecting to and managing Microsoft Fabric semantic models (formerly Power BI datasets). It allows client applications to query these models using languages like DAX and MDX, not for running T-SQL queries against raw files in OneLake.

Correct answer: B — The Manage fields operator

• A. Incorrect. The Filter operator is used to include or exclude events based on specific conditions. It is useful for reducing the volume of data but does not provide functionality to modify the schema or add new calculated columns.
• B. Correct. The Manage fields operator is specifically designed for schema modification. It allows you to add, remove, or rename fields, and critically, to create new calculated fields using expressions. This makes it the ideal choice for adding a processing timestamp column to enrich the data before it is ingested into the KQL database.
• C. Incorrect. The Group by operator is used for performing aggregations on streaming data over a time window. While it transforms the data, it changes its granularity and is not intended for altering the schema of individual events by adding a simple column.
• D. Incorrect. The Route operator is used to direct events to different destinations or paths based on specified rules or conditions. It does not alter the structure or content of the events themselves.

Correct answer: A — Data ingestion mapping

• A. Correct. When ingesting data into a KQL Database (which uses the Kusto engine), a 'Data ingestion mapping' is used to map source data fields, such as properties in a JSON object, to the columns of the target table. This mapping is a crucial setting within the Eventstream's KQL Database destination configuration to handle schema mismatches between the source data and the target table.
• B. Incorrect. An Event Processor is a component within the Eventstream itself that allows for custom, in-flight processing of data *before* it is sent to a destination. While it could be used to transform data, it is not the specific setting within the KQL Database destination used for mapping source fields to table columns.
• C. Incorrect. An update policy is a KQL table-level object that automatically runs a transformation query on newly ingested data in a source table to modify and append the results to a target table. It is configured on the KQL table itself, not in the Eventstream destination settings, and is used for post-ingestion transformations.
• D. Incorrect. Mirroring is a feature for creating a continuously synchronized copy of a database, typically for replication or disaster recovery. It does not perform field-level data mapping during ingestion.

Correct answer: A — parse_json(Properties).device.id

• A. Correct. The `parse_json` function, which is an alias for `todynamic`, converts a string containing JSON into a `dynamic` data type. Once the string is parsed into a dynamic object, you can use dot notation to navigate the object's properties. The expression `parse_json(Properties).device.id` correctly parses the JSON and then accesses the nested `id` property within the `device` object. This is the idiomatic and most common way to extract values from JSON in KQL.
• B. Incorrect. Kusto Query Language (KQL) does not have a function named `extract_json`. The actual function for this purpose is `extractjson` (without an underscore), and its syntax is `extractjson(jsonPath, dataSource, type)`, which differs from the provided option in both name and parameter order.
• C. Incorrect. The `split` function is used for basic string manipulation by breaking a string into an array of substrings based on a delimiter. It is not aware of JSON structure and cannot reliably parse JSON data, especially with nested objects, arrays, or strings that might contain the delimiter character.
• D. Incorrect. While the `todynamic` function is the correct function to parse the JSON string, this option is incomplete. It only performs the conversion of the `Properties` column to a dynamic object but does not include the subsequent step of accessing the `device.id` property. The complete expression would be `todynamic(Properties).device.id`, which is functionally equivalent to option A.

Correct answer: A — Delete data for the specific partition (Date) before inserting the new data.

• A. This is the correct pattern. Deleting the data for a specific partition (in this case, by date) before inserting the corrected data is a classic idempotent loading strategy. This ensures that each re-run completely replaces the data for that specific partition, resulting in the same final state regardless of how many times the process is executed. This is an efficient and common approach for partitioned fact tables.
• B. This approach is incorrect because using `INSERT INTO` without any pre-check or post-cleanup will append new rows on every run. If a load for a specific date is re-run, it will result in duplicate data, which violates the principle of idempotency.
• C. This is incorrect. Using `mode("append")` in a Spark write operation is functionally equivalent to a simple `INSERT INTO` statement. It adds data to the table without checking for existing records, which will lead to data duplication if the load is re-run for the same date. An idempotent alternative in Spark would be `mode("overwrite")`, but that typically replaces the entire table or specific partitions dynamically.
• D. This is an unreliable and incomplete pattern for ensuring idempotency in a data warehouse context. While a primary key can prevent duplicate rows, it often causes the load to fail upon encountering a duplicate. This doesn't achieve the goal of replacing corrected data. Furthermore, many modern analytical data platforms either do not enforce primary key constraints in the same way as OLTP systems or doing so can cause significant performance issues, making it an unsuitable strategy for large-scale data loading.

Correct answer: C — Dataflow Gen2.

• A. Incorrect. A Warehouse T-SQL Script is a code-based approach. While you can implement conditional logic and data validation using T-SQL, it requires writing code and does not provide the no-code, drag-and-drop visual interface specified in the question.
• B. Incorrect. A Spark Notebook with PySpark is a powerful, code-centric tool for complex data transformations. However, it requires writing PySpark code and is not a no-code visual tool for implementing ETL logic.
• C. Correct. Dataflow Gen2 leverages the Power Query experience in Fabric, which offers a no-code/low-code visual data transformation canvas. It is specifically designed for visual ETL and data quality patterns, allowing users to easily implement conditional split logic to route valid rows to one destination (e.g., a 'Clean' table) and invalid rows to another (e.g., an 'Error' table) without writing code.
• D. Incorrect. A Real-Time Dashboard is a visualization tool used for monitoring and displaying data. It is not designed for data ingestion, transformation (ETL), or implementing data quality and splitting logic.

Correct answers: A, B — Set `.option("mergeSchema", "true")` during the write operation.; Set `spark.databricks.delta.schema.autoMerge.enabled = true` in the Spark configuration.

• A. This is a correct answer. Setting the DataFrameWriter option `.option("mergeSchema", "true")` during a write operation (e.g., in `append` or `overwrite` mode) instructs Delta Lake to compare the schema of the data being written with the schema of the target table. If new columns are found in the source DataFrame, they will be automatically added to the target table's schema, thus enabling schema evolution.
• B. This is a correct answer. Setting the Spark session configuration `spark.databricks.delta.schema.autoMerge.enabled = true` enables automatic schema evolution specifically for `MERGE INTO` operations. When this configuration is active, a merge operation can add new columns from the source to the target Delta table if they don't already exist.
• C. This is incorrect. The `.option("overwriteSchema", "false")` setting explicitly prevents the schema from being changed during an overwrite operation. If you were to set `.option("overwriteSchema", "true")`, it would completely replace the existing schema with the new DataFrame's schema, rather than merging them by adding new columns.
• D. This is incorrect in the context of the best two answers. While the `COPY INTO` command is a valid way to load data and can be configured to handle schema evolution using `COPY_OPTIONS ('mergeSchema' = 'true')`, options A and B represent the direct, idiomatic configurations for handling schema evolution within standard PySpark DataFrame `write` and `merge` operations, which are central to the scenario described.

Correct answer: B — Use a Data Factory pipeline running every 10 minutes to delete duplicates.

• A. Incorrect. An Update Policy in a KQL Database processes data as it is ingested, transforming it from a source to a target table in near-real-time. This mechanism does not support buffering data for a 10-minute window to wait for potential late-arriving duplicates. The policy acts on data immediately, which does not meet the specified windowing requirement.
• B. Correct. This batch-oriented approach directly implements the required 10-minute processing window. A scheduled Data Factory pipeline can run every 10 minutes to process raw data from a staging table. Each run would execute a KQL query to deduplicate events within that time slice (e.g., using `summarize arg_max(...) by EventId`) and load the clean data into a final table for analysis. This pattern precisely fulfills the requirement of deduplicating within a specific time window before making the data available.
• C. Incorrect. There is no built-in 'Drop Duplicates' feature on a Fabric Eventstream input that provides custom, time-windowed deduplication based on a specific data field. Such logic must be implemented in a subsequent processing or transformation step.
• D. Incorrect. While a materialized view is an excellent mechanism for real-time deduplication in KQL, this option is flawed. The `distinct` operator is not the correct tool for this type of key-based deduplication; the `arg_max()` aggregation function is required. Furthermore, a materialized view provides continuous, near-real-time deduplication, not a discrete 10-minute batch window as implied by the requirement.

Correct answer: A — Shortcut caching

• A. Shortcut caching is a performance optimization feature for OneLake shortcuts that caches data from external sources like Amazon S3 within the Fabric workspace. This reduces egress costs and improves query performance by serving subsequent reads from the local cache, avoiding repeated fetches from the original source.
• B. There is no 'Materialized Shortcut' feature. Materialized views exist in Azure Synapse Analytics and indexed views in Azure SQL Database, but they pre-compute query results rather than caching shortcut data. Shortcut caching is the correct term for the performance-enhancing cache.
• C. Cached View is not a Fabric performance feature. While views are logical query constructs, caching in Fabric specifically applies to OneLake shortcuts (shortcut caching) to improve data access speed, not to views.
• D. Mirroring in Fabric refers to replicating entire databases (e.g., mirroring Azure SQL Database to Fabric) for near real-time analytics. It is unrelated to caching for individual shortcuts. Shortcut caching is the dedicated feature for improving performance of data referenced through OneLake shortcuts.

Correct answer: A — Executors

• A. Correct. The Executors tab in the Spark UI is the primary location for diagnosing executor-level issues. It lists each executor and provides detailed information including its status, memory usage (JVM heap), disk usage, and the number of active tasks. Crucially, it provides direct links to the stdout and stderr logs for each executor, which would contain the specific 'OutOfMemoryError' stack trace, definitively identifying the failed executor.
• B. Incorrect. The Environment tab displays configuration details for the Spark application, such as Spark properties, system properties, and classpath information. It is useful for verifying configuration settings but does not provide runtime metrics or error logs for individual executors.
• C. Incorrect. The Storage tab shows details about RDDs and DataFrames that have been cached in memory or on disk. While this is related to memory, it focuses on data caching rather than the overall JVM heap memory of an executor. It won't directly identify which executor failed due to an 'OutOfMemoryError'.
• D. Incorrect. The SQL tab is dedicated to monitoring and analyzing queries executed via Spark SQL. It provides information about query plans, stages, and performance metrics for SQL operations, but it does not report on the health or memory status of individual executors.

Correct answers: A, D — Increase the number of partitions in the source Event Hub to match the parallelism of the Eventstream.; Scale the Fabric Capacity to provide more Compute Units (CUs).

• A. Correct. The degree of parallelism for an Eventstream processor is directly tied to the number of partitions in the source Event Hub. By increasing the number of partitions, the Eventstream can read and process data from multiple partitions concurrently. This distributes the load, improves overall throughput, and reduces ingestion latency, which is a fundamental strategy for scaling out the ingestion pipeline.
• B. Incorrect. Reducing the number of partitions to one would create a significant performance bottleneck. It forces all data to be processed serially through a single channel, eliminating parallelism. This would dramatically decrease throughput and increase ingestion latency, making it the opposite of the desired optimization.
• C. Incorrect. Consumer groups provide independent, logical views of the event stream for different consuming applications. An Eventstream uses a single consumer group to manage its state. Changing the consumer group or the non-existent concept of a 'larger' consumer group does not increase the processing resources or throughput allocated to the Eventstream itself. Performance is dictated by parallelism (partitions) and compute resources (CUs).
• D. Correct. The Eventstream processor consumes Microsoft Fabric Capacity, measured in Compute Units (CUs). CUs represent the processing power (CPU and memory) available. If the current capacity is insufficient for the data volume, the Eventstream job will be throttled, leading to backpressure and increased ingestion latency. Scaling the Fabric Capacity provides more CUs, directly allocating more resources to the processor, allowing it to handle the load more efficiently and reduce latency.

Correct answer: D — Use the built-in debugger (experimental) or check driver logs (StdOut).

• A. Incorrect. The Spark History Server is designed for analyzing the performance and execution of Spark jobs at an application level. It provides metrics, stage details, executor information, and job-level stack traces, but it does not capture the specific values of Python variables within the notebook's code at the moment of a crash.
• B. Incorrect. The 'Variables' explorer pane is a tool for inspecting the state of variables during a live, interactive notebook session. Its scope is limited to the current active session and cannot be used to retrospectively examine the variable state of a previous, crashed run.
• C. Incorrect. While adding print() statements is a common debugging technique, it requires modifying the code and rerunning the notebook. Since the problem states the error cannot be reliably reproduced, this method is inefficient and may not succeed in capturing the necessary information.
• D. Correct. The driver logs (StdOut/StdErr) are the best resource for post-mortem analysis of a failed notebook run. These logs capture the output from the Spark driver process, including any unhandled exceptions, full stack traces, and any output printed immediately before the crash. This allows you to inspect the state and potential variable values exposed in the error message without needing to reproduce the issue. The built-in debugger is also a powerful tool for live debugging when an issue can be reproduced.

Correct answers: A, C — The `Data insights` and `Runtime logs` tabs to view throughput and potential errors.; The `Data preview` tab to see live data snapshots.

• A. Correct. The Eventstream editor provides two key monitoring tabs. The `Data insights` tab displays performance metrics like `IncomingMessages` and `OutgoingMessages`, which measure throughput. The `Runtime logs` tab shows detailed operational logs, including warnings and errors, which are crucial for identifying processing issues that could lead to dropped events.
• B. Incorrect. The SQL analytics endpoint is used to query data *after* it has been successfully ingested and stored in the Lakehouse. It is a downstream verification method and not a feature within the Eventstream editor for monitoring the real-time data flow or diagnosing in-flight processing errors.
• C. Correct. The `Data preview` tab is a built-in feature of the Eventstream editor that allows you to view a live snapshot of the event data as it flows through the source, the eventstream itself, or into a destination. This is a primary method for visually confirming that data is being ingested and structured correctly.
• D. Incorrect. The `Capacity Metrics` app is a workspace-level tool used to monitor overall Fabric capacity consumption and identify throttling. While performance issues in Eventstream can be related to capacity, this app is separate from the Eventstream editor and is used for broader performance analysis, not for direct data flow verification or error identification within a specific stream.

Correct answer: A — To visualize the dependencies and data flow between items to understand impact analysis.

• A. Correct. The Lineage view is a visualization tool that shows dependencies and data flow between items in a Fabric workspace. Its primary purpose is to help users understand relationships, trace data from source to destination, and perform impact analysis.
• B. Incorrect. Monitoring live CPU usage and real-time resource consumption is a function of the Fabric Capacity Metrics app or Monitoring Hub, not the Lineage view. The Lineage view focuses on data dependencies and relationships, not resource utilization.
• C. Incorrect. Configuring alert rules for pipeline failures or data quality issues is done through the Monitoring Hub or other monitoring services. The Lineage view is a read-only visualization tool for data dependencies and does not support alert configuration.
• D. Incorrect. Editing SQL code to modify transformation logic or troubleshoot query performance is performed in item-specific editors like the SQL query editor. The Lineage view is strictly for visualizing relationships and data flow, not for code editing.

Correct answers: A, B — The SQL Endpoint auto-sync process has failed or is pending due to resource constraints.; The Delta table was written with a transaction log version not yet supported by the SQL Endpoint.

• A. Correct. The SQL Endpoint auto-sync process is responsible for reflecting Delta table changes in the SQL analytics endpoint. If this process fails or is pending due to resource constraints, the endpoint will serve stale data until synchronization completes.
• B. Correct. If the Delta table was written with a transaction log version that uses features not yet supported by the SQL Endpoint's reader, the endpoint cannot parse the new log entries. This version mismatch prevents the SQL Endpoint from recognizing the latest data.
• C. Incorrect. The Spark job successfully wrote the data, indicating the writing principal had appropriate permissions. The user's inability to see data is a read-side issue; lacking 'Write' permissions does not cause synchronization lag for the SQL Endpoint.
• D. Incorrect. In Microsoft Fabric, a Lakehouse and its capacity are always in the same region. A region mismatch is architecturally impossible and cannot be the cause of synchronization delays.
• E. Incorrect. The question states the data was written to a Delta table. Writing as CSV contradicts this premise and would not involve the Delta transaction log, so it cannot explain a lag in the SQL Endpoint reflecting Delta table updates.

Correct answer: B — Set the 'Batch count' property in the 'For Each' activity to a lower number (e.g., 5).

• A. Incorrect. Enabling 'Secure Input' on the Notebook activity is a security feature that prevents sensitive parameter values from being logged. It does not affect the number of concurrent jobs submitted to the Spark pool and will not resolve throttling errors.
• B. Correct. The 'For Each' activity runs iterations in parallel by default. Setting the 'Batch count' property to a lower number, such as 5, limits the number of concurrent loop iterations, thereby reducing the number of simultaneous Spark jobs and preventing throttling while still allowing parallel processing.
• C. Incorrect. Increasing the retry count and retry interval does not address the root cause of exceeding the concurrent job limit. The Notebook activity will still fail due to throttling, and subsequent retries are likely to encounter the same error, potentially increasing system load.
• D. Incorrect. Switching the 'For Each' activity to sequential execution would process only one file at a time, which avoids throttling but eliminates all parallelism. This significantly increases total pipeline execution time, making it a less efficient solution compared to adjusting the 'Batch count'.

Correct answer: B — The capacity has exceeded its burstable Compute Units (CUs) limits over the smoothing period.

• A. Incorrect. The 'Throttling' metric indicates compute resource enforcement, not network latency. Network issues would appear as increased latency or failed calls, not as throttling.
• B. Correct. A non-zero 'Throttling' metric means the capacity has exceeded its burstable Compute Units (CUs) limits over the smoothing period. This triggers throttling, delaying interactive operations to bring consumption back within limits.
• C. Incorrect. Throttling relates to compute (CU) usage, not storage. Reaching the OneLake storage limit would cause write failures and appear in storage metrics, not the throttling metric.
• D. Incorrect. Throttling manages compute consumption, not user licensing. Exceeding license limits is an entitlement issue and would not be reflected in the performance-related throttling metric.

Correct answer: B — Increase the 'Batch count' property in the ForEach activity settings.

• A. Incorrect. Enabling 'Sequential' in the ForEach settings forces the activity to process items one at a time, eliminating all parallelism. This would further increase execution time, which is the opposite of the desired outcome.
• B. Correct. The 'Batch count' property controls the maximum number of concurrent iterations in a ForEach activity. Increasing it allows more tables to be copied in parallel, up to the available capacity, resolving the bottleneck of only 4 concurrent copies.
• C. Incorrect. The 'Retry count' on the inner Copy activity specifies how many times a failed copy is retried. This setting improves fault tolerance but does not affect the number of parallel executions.
• D. Incorrect. A Switch activity is used for conditional branching based on a value, not for iterating over a list. It does not provide parallel processing and cannot replace a ForEach for copying multiple tables.

Correct answer: B — There is data skew. Perform a Broadcast join on the smaller lookup table.

• A. Incorrect. A driver running out of memory typically causes the entire job to fail or hang, often with OutOfMemoryError exceptions in the driver logs. It would not manifest as a single, long-running task while all others complete quickly. The symptom described points to an imbalance in data distribution among executor tasks, not a driver-side memory issue.
• B. Correct. A single task taking significantly longer than its peers is a classic symptom of data skew. This occurs when the data is not partitioned evenly across the join keys, causing one partition (and thus one task) to receive a disproportionately large amount of data to process. For a join between a large and a small table, a broadcast join is the ideal solution. It sends a copy of the smaller table to every executor, allowing the join to be performed locally without shuffling the large table, which effectively bypasses the data skew problem.
• C. Incorrect. CPU starvation would lead to a general slowdown across most or all tasks, not an isolated straggler task. While adding more cores might speed up the overall job, it would not resolve the root cause of the imbalance where one task has much more work to do than the others.
• D. Incorrect. The issue is related to data distribution during processing, not the efficiency of the on-disk storage format. Parquet is a highly efficient columnar format optimized for analytical queries. Switching to Avro, a row-based format, would not address data skew and could potentially degrade performance for this type of workload.