Free AWS Certified Data Engineer - Associate (DEA-C01) Sample Questions

Correct answers: B, C — AWS CodePipeline; AWS CodeBuild

• A. Incorrect. Amazon SageMaker is a fully managed service for building, training, and deploying machine learning (ML) models. It is not designed for creating CI/CD pipelines for general application code or ETL scripts.
• B. Correct. AWS CodePipeline is a fully managed continuous delivery service that orchestrates and automates the entire software release process. It is essential for defining the stages of the pipeline (source, build, test, deploy) and connecting the various services, such as triggering the pipeline from a CodeCommit push.
• C. Correct. AWS CodeBuild is a fully managed continuous integration service that compiles source code, runs tests, and produces software packages that are ready to deploy. In this scenario, CodeBuild would be a stage within CodePipeline responsible for running unit tests or validation checks on the Glue ETL scripts before they are deployed.
• D. Incorrect. Amazon EC2 provides raw virtual server compute capacity. While you could manually set up CI/CD tools on an EC2 instance, it is not a managed service specifically designed for this purpose. For a standard AWS CI/CD solution, CodePipeline and CodeBuild are the essential, purpose-built services.
• E. Incorrect. AWS Data Pipeline is a service used to orchestrate and automate data movement and transformation workflows (e.g., scheduling data copies or EMR jobs). It is designed for data orchestration, not for software delivery (CI/CD) pipelines that deploy code changes.

Correct answer: B — To enable parallel processing across multiple nodes in a cluster.

• A. Incorrect. Data partitioning is a mechanism for distributing data for processing, whereas data encryption at rest is a security measure. While both can be applied to the same dataset, their purposes are fundamentally different and unrelated.
• B. Correct. The fundamental purpose of partitioning in distributed frameworks like Apache Spark is to divide a large dataset into smaller, manageable chunks (partitions). These partitions can be distributed across multiple worker nodes in a cluster and processed in parallel, which significantly improves performance, scalability, and overall computational efficiency.
• C. Incorrect. Data compression is a technique used to reduce the physical size of data, which helps in minimizing storage costs and network I/O. While often used in distributed systems, it is a separate optimization from partitioning, which is concerned with distributing the workload for parallel execution.
• D. Incorrect. Creating backup copies for disaster recovery is a data management and resilience strategy. Data partitioning, in the context of Spark, is about organizing data for efficient parallel computation, not for creating redundant copies for recovery purposes.

Correct answer: B — An AWS Glue ETL job scheduled to run daily.

• A. Incorrect. While AWS Lambda is a serverless service, it is not designed for large-scale, long-running ETL jobs. Processing 50 GB of data would likely exceed Lambda's maximum execution time (15 minutes) and memory limits, making it unsuitable and inefficient for this use case.
• B. Correct. AWS Glue is a fully managed, serverless ETL service specifically designed for large-scale data processing. It can be scheduled to run daily, efficiently handle 50 GB of data, perform the conversion from CSV to Parquet, partition the data, and automatically update the AWS Glue Data Catalog. This seamless integration with Athena makes it the most cost-effective and efficient solution that meets all requirements.
• C. Incorrect. Amazon EMR is a powerful big data processing service, but it is not a serverless solution in its traditional form. It requires managing a cluster of servers, which incurs higher operational overhead and is generally more expensive than AWS Glue for a scheduled daily ETL job of this scale.
• D. Incorrect. Using an Amazon EC2 instance is not a serverless approach. This option requires manual management of the instance, including patching, scaling, and ensuring fault tolerance. It is the least efficient and most operationally complex solution compared to a managed serverless service like AWS Glue.

Correct answer: C — Amazon Kinesis Data Analytics for Apache Flink

• A. Incorrect. AWS Lambda is a serverless, event-driven compute service. While it can process data from Kinesis Data Streams, it is inherently stateless. Implementing complex, stateful processing required for fraud detection would necessitate an external state management service, making it less ideal than a service specifically designed for stateful stream processing.
• B. Incorrect. AWS Glue streaming ETL is designed for continuous extract, transform, and load (ETL) jobs, primarily for loading data into data lakes or warehouses. It is not optimized for the low-latency, complex event processing and stateful analytics required for a near-real-time fraud detection system.
• C. Correct. Amazon Kinesis Data Analytics for Apache Flink is a purpose-built service for building and managing applications for stateful, real-time stream processing. It can process data from Kinesis Data Streams, maintain state over time (e.g., transaction windows), and enrich the stream with reference data from sources like Amazon S3, making it the ideal choice for this use case.
• D. Incorrect. Amazon Simple Queue Service (SQS) is a message queuing service used to decouple and buffer messages between application components. It does not have any native capabilities for data processing, transformation, or stateful analytics.

Correct answer: C — AWS Glue workflows

• A. Incorrect. While Amazon MWAA is a powerful managed orchestration service for Apache Airflow and can orchestrate Glue jobs, it is a separate service and not a feature within the AWS Glue ecosystem itself. The requirement is for a tool within Glue, making this option less suitable than the native solution.
• B. Incorrect. AWS Step Functions is a versatile serverless orchestration service that can coordinate multiple AWS services, including Glue jobs, using a visual workflow builder. However, it is a separate, general-purpose service. The question specifically asks for a tool within the AWS Glue service ecosystem, for which AWS Glue Workflows is the purpose-built solution.
• C. Correct. AWS Glue workflows is a native, managed feature within AWS Glue that provides a visual, graph-based interface to create, run, and monitor complex ETL workflows. It is specifically designed to orchestrate AWS Glue crawlers, jobs, and triggers, allowing users to define dependencies and run tasks in parallel. This perfectly matches all the requirements of the scenario.
• D. Incorrect. AWS Batch is a service for scheduling and running batch computing workloads on AWS. It focuses on managing compute resources for batch jobs but does not provide a high-level, visual orchestration framework for defining dependencies between different steps in a data pipeline, such as between Glue crawlers and ETL jobs.

Correct answers: B, C — Amazon S3 event notifications; AWS Step Functions

• A. Incorrect. Amazon EC2 is a server-based compute service (IaaS) and is not serverless. Using EC2 would require provisioning and managing servers, which directly contradicts the core requirement of a 'completely serverless' pipeline.
• B. Correct. Amazon S3 event notifications are the ideal mechanism for initiating an event-driven workflow. They can be configured to automatically trigger a downstream process, such as an AWS Step Functions state machine or an AWS Lambda function, whenever a new object like a JSON file is created in an S3 bucket. This serves as the serverless entry point for the pipeline.
• C. Correct. AWS Step Functions is a serverless orchestration service specifically designed to build and manage multi-step workflows. It allows you to coordinate multiple AWS services (like AWS Lambda for validation and transformation) into a state machine, manage the execution state, handle errors, and provide a visual representation of the workflow. This perfectly aligns with the requirement for an 'orchestrated' and 'state managed' process.
• D. Incorrect. Amazon RDS is a managed relational database service. It is used for data storage and retrieval, not for triggering events or orchestrating data pipelines. It plays no role in the control flow of a serverless workflow.
• E. Incorrect. While Amazon MWAA (Managed Workflows for Apache Airflow) is an orchestration service, it is not considered fully serverless in the same way as AWS Step Functions. MWAA is better suited for complex, scheduled batch ETL jobs, whereas Step Functions is the idiomatic and foundational choice for building lightweight, event-driven, and completely serverless workflows on AWS due to its tight integration with services like Lambda and S3 events.

Correct answer: A — Increase the number of DPUs (Data Processing Units) allocated to the job.

• A. This is the correct answer. A Data Processing Unit (DPU) in AWS Glue provisions processing resources (vCPU and memory). The number of Spark executors a job can use is determined by the number of DPUs allocated. The problem states the bottleneck is a single executor. By increasing the number of DPUs, the job gets more executors. If the large compressed file uses a splittable compression format (like bzip2) or is uncompressed, Spark can read and process different parts of the file in parallel across these multiple executors. This directly addresses the stated problem by enabling parallel processing through proper resource allocation.
• B. Incorrect. While splitting the file would enable parallel processing, it is not the best practice and is not a direct configuration of the Glue job itself. It adds an extra pre-processing step, increasing operational overhead and complexity. The preferred approach is to configure the Glue job with adequate resources to handle large, splittable files directly.
• C. Incorrect. This option misunderstands the purpose of `groupFiles`. The `groupFiles` and `groupSize` options are designed to solve the 'small files problem' by combining many small input files into fewer, larger partitions. This improves efficiency when the source contains thousands of small files. It does not split a single large file; it does the opposite.
• D. Incorrect. An AWS Glue 'Spark' job is designed for distributed, large-scale data processing. A 'Python Shell' job runs on a single node and is intended for smaller tasks and scripts that don't require a distributed framework. Migrating a 50 GB file process to a Python Shell job would remove all parallelism and drastically decrease performance, likely causing the job to fail from insufficient memory.

Correct answer: C — An Amazon EventBridge Scheduler schedule.

• A. Incorrect. Using a cron job on an Amazon EC2 instance is not a serverless approach. This method requires provisioning and managing the underlying server, which adds operational overhead and contradicts the requirement for a serverless solution.
• B. Incorrect. While an EventBridge scheduled rule can trigger a Glue job, AWS documentation refers to this as a legacy feature. For new implementations, AWS explicitly recommends using Amazon EventBridge Scheduler, which offers greater scalability and more advanced features, making it the more suitable choice.
• C. Correct. Amazon EventBridge Scheduler is the recommended, serverless service for creating, running, and managing scheduled tasks at scale. According to AWS documentation, it is the preferred solution for invoking targets on a schedule. It supports cron expressions and can directly invoke an AWS Glue job, making it the most suitable and modern option for this use case.
• D. Incorrect. While AWS Step Functions can be triggered on a schedule, it is designed for orchestrating multi-step workflows. Using it for a single, simple scheduled task is overly complex and not as direct as using the purpose-built Amazon EventBridge Scheduler. A `Wait` state is used to pause an *existing* workflow, not to schedule its initial invocation.

Correct answer: B — Apache Hive Metastore

• A. Incorrect. Apache YARN (Yet Another Resource Negotiator) is the resource management and job scheduling component of the Hadoop ecosystem. Its function is to allocate system resources to various applications and schedule tasks, which is fundamentally different from the metadata cataloging role of the AWS Glue Data Catalog.
• B. Correct. The AWS Glue Data Catalog is specifically designed to be a managed, Hive-compatible, drop-in replacement for the Apache Hive Metastore. It serves as a central repository for storing structural and operational metadata for data assets, such as table definitions, schemas, and partition information, making this metadata available to various AWS services like Athena, EMR, and Redshift Spectrum.
• C. Incorrect. Apache HDFS (Hadoop Distributed File System) is the primary storage system used by Hadoop applications. It is responsible for storing the actual data across a cluster. The AWS Glue Data Catalog stores metadata about the data, not the data itself. Amazon S3 is the AWS equivalent storage service often used with the Glue Data Catalog.
• D. Incorrect. Apache ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services for distributed applications. It acts as a coordination service, which is a different function from the metadata management provided by the AWS Glue Data Catalog.

Correct answer: B — Resource Links

• A. Incorrect. Table Versions in the AWS Glue Data Catalog are used to track and manage changes to a table's schema over time. This feature does not provide a mechanism for establishing relationships or semantic links between different tables.
• B. Correct. Resource Links in AWS Glue are pointers to databases or tables. While a primary use case is sharing data across accounts, they can also be used to create logical, semantic links between tables within the same account. This allows you to represent relationships between data resources, making it the most suitable option among the choices for linking the `customers` and `orders` tables.
• C. Incorrect. Partitions are used to divide a single table's data into smaller, more manageable pieces based on the values of specific columns (e.g., date, region). This is an optimization technique to improve query performance and reduce costs by scanning less data, but it does not define relationships between different tables.
• D. Incorrect. Table Properties allow you to store user-defined key-value metadata about a table. While you could add a property to describe a relationship in plain text, this does not create a formal, queryable, or semantic link that AWS services can understand. It is simply descriptive metadata and not a functional link.

Correct answer: B — It minimizes data redundancy and improves data integrity.

• A. Incorrect. Normalization often complicates analytical queries rather than simplifying them. By splitting data into multiple tables, it necessitates complex JOIN operations to retrieve a complete view of the data, which is typical for reporting. Analytical systems (OLAP) often use denormalized models like star schemas to optimize query performance.
• B. Correct. The fundamental purpose of normalization is to reduce data redundancy by ensuring that each piece of data is stored in only one place. This elimination of redundancy prevents data anomalies (insert, update, and delete anomalies), thereby significantly improving data integrity and consistency. These characteristics are paramount for OLTP systems, which handle frequent, small transactions and require high accuracy.
• C. Incorrect. A highly normalized schema can degrade read performance for large-scale aggregations due to the need for multiple, often costly, JOIN operations across tables. Data models optimized for analytical read performance, such as denormalized schemas or columnar storage, are better suited for these workloads.
• D. Incorrect. Normalized data models are designed for structured, relational data with a predefined schema. They are not intended to natively handle semi-structured (e.g., JSON, XML) or unstructured data. Specialized databases, such as NoSQL databases or data lakes, are better equipped for managing these data types.

Correct answer: B — DynamoDB is schemaless, only requiring a primary key; individual items in the same table can have different sets of attributes.

• A. Incorrect. This describes a relational database model. Amazon DynamoDB is a NoSQL database that does not require a strict, predefined schema for all items in a table. It only mandates the definition of the primary key attributes.
• B. Correct. DynamoDB is fundamentally schemaless, except for the mandatory primary key (which includes a partition key and an optional sort key). This design allows for maximum flexibility, as each item in a table can have a different structure and set of attributes. This facilitates agile development and easy schema evolution over time.
• C. Incorrect. While AWS Glue Data Catalog and AWS Lambda can be used in data processing pipelines that involve DynamoDB, they do not natively define or enforce the schema for a DynamoDB table itself. DynamoDB's schema enforcement is handled internally and is limited to the uniqueness and data types of its primary key.
• D. Incorrect. `ALTER TABLE` is a command used in SQL-based relational databases to modify a table's structure. DynamoDB, as a NoSQL database, does not use this command. Schema evolution in DynamoDB is managed by adding or removing attributes at the individual item level, without requiring table-level schema modification commands.

Correct answer: C — In the S3 console, show all versions, then delete the current version (which is the overwritten one) to expose the previous version.

• A. This is incorrect. The S3 Restore API is used specifically to restore objects that have been archived to the S3 Glacier or S3 Glacier Deep Archive storage classes. The question does not state that the previous version was transitioned to Glacier; it is simply a non-current version in the bucket.
• B. This describes a valid, but less direct, method for recovery. You can indeed copy a previous version of an object to the same key in the same bucket. This action creates a new version of the object that is a copy of the old one, effectively making the desired content the 'current' version again. However, this adds another object version to the stack rather than simply reverting the last change.
• C. This is the most direct and correct method to recover the previous version. When S3 versioning is enabled, overwriting a file creates a new version, making it the 'current' version. By deleting the specific version ID of this new, current version, you effectively remove it. S3 then automatically promotes the next most recent version to become the current version, thus recovering the original file's state before the overwrite.
• D. This is incorrect. S3 Cross-Region Replication (CRR) is a proactive feature for disaster recovery and data distribution that must be configured before the data modification occurs. It cannot be enabled retroactively to recover a previous version of an object that has already been overwritten.

Correct answer: C — S3 Cross-Region Replication (CRR)

• A. S3 Versioning is incorrect. This feature maintains multiple versions of an object in the same bucket, which protects against accidental overwrites or deletions. However, it does not replicate data to a different AWS region and therefore cannot provide disaster recovery in the case of a regional outage.
• B. S3 Lifecycle Policies are incorrect. These policies are used to automate the management of an object's lifecycle, such as transitioning data to different storage classes (e.g., S3 Standard to S3 Glacier) or expiring old objects to manage costs. They do not replicate data across regions for disaster recovery.
• C. S3 Cross-Region Replication (CRR) is the correct solution. CRR is specifically designed to automatically and asynchronously replicate objects from a source bucket in one AWS region to a destination bucket in a different region. Configuring CRR from `us-west-2` to `eu-central-1` meets the disaster recovery requirement by ensuring a copy of the data is available in a separate geographic location in the event of a primary region failure.
• D. S3 Same-Region Replication (SRR) is incorrect. SRR replicates objects to another bucket within the same AWS region. While useful for specific use cases like aggregating logs or maintaining separate replicas for testing, it does not protect against a complete regional outage, which is the core requirement of the question.

Correct answers: B, D — Parquet; Snappy

• A. Incorrect. JSON is a row-based format. For queries that only access a subset of columns, Athena must still scan entire rows of data, making it inefficient. This increases the amount of data scanned, leading to higher costs and slower query performance compared to columnar formats.
• B. Correct. Apache Parquet is a columnar storage format optimized for analytical workloads. When a query needs only a few columns, Athena can read just the data for those specific columns, drastically reducing the amount of data scanned from S3. This directly translates to lower costs and significantly better query performance.
• C. Incorrect. CSV is a simple, row-based text format. Similar to JSON, it is not optimized for analytical queries on specific columns, as the entire file must be scanned. This results in high data scanning costs and poor performance for the described use case.
• D. Correct. Snappy is a compression algorithm that provides a good balance between compression ratio and decompression speed, making it ideal for performance-sensitive analytical queries. It is also a 'splittable' compression format, which allows Athena to read chunks of a file in parallel, further improving query performance. It is commonly used with Parquet.
• E. Incorrect. Using uncompressed files results in larger data sizes on S3, which increases storage costs. More importantly for Athena, it maximizes the amount of data that needs to be read from S3 for each query, leading to the highest possible data scanning costs and the slowest query performance.

Correct answer: B — Concurrency Scaling

• A. Incorrect. Elastic Resize is used to change the number or type of nodes in a Redshift cluster. While it scales capacity, it is typically a manual or scheduled process, not an automatic and transparent response to a sudden spike in query load. It also involves a brief downtime or read-only period, making it unsuitable for handling transient, peak-hour traffic.
• B. Correct. Concurrency Scaling is the specific Redshift feature designed to handle this exact scenario. It automatically and transparently adds transient cluster capacity to manage sudden spikes in concurrent read queries. This ensures consistent performance during peak times by offloading queries to new clusters, reducing queue times without requiring any manual intervention.
• C. Incorrect. Materialized Views improve query performance by pre-computing and storing the results of complex and frequently run queries. While they can make individual queries faster, they do not add compute capacity to the cluster or address issues arising from a high number of concurrent users.
• D. Incorrect. Workload Management (WLM) is used to manage and prioritize queries on the existing cluster resources. It allows you to define query queues and allocate resources, but it does not add any new compute capacity. It manages how existing resources are used, rather than scaling them to meet increased demand.

Correct answer: C — Validating the `email_address` column against a regular expression.

• A. Incorrect. Checking for null values only confirms the presence of data in the column. While it is a useful data quality check, it does not validate whether the existing string conforms to a valid email format. A non-null but incorrectly formatted string like 'not-an-email' would pass this check.
• B. Incorrect. Verifying the data type (e.g., ensuring it is a string) is a fundamental data quality step, but it is too generic for this specific requirement. It confirms the data is of the correct type but does not enforce the structural pattern of a valid email address (e.g., user@domain.com).
• C. Correct. A regular expression (regex) is a sequence of characters that specifies a search pattern. It is the most precise and suitable method for validating that a string conforms to a specific format, such as that of an email address. AWS Glue Data Quality rules or custom logic within a Glue script can leverage regex to enforce this business requirement effectively.
• D. Incorrect. Ensuring uniqueness is a data quality check that prevents duplicate values within the `email_address` column. This is an important rule for data integrity (e.g., ensuring one account per email), but it is unrelated to validating the format of the individual email strings themselves.

Correct answers: B, D — AWS Glue Data Quality; Data Quality Definition Language (DQDL)

• A. Incorrect. AWS Glue Crawlers are used to connect to data stores, scan the data, infer schemas, and populate the AWS Glue Data Catalog. Their primary function is metadata discovery, not the definition or enforcement of data quality rules.
• B. Correct. AWS Glue Data Quality is the specific AWS feature designed to measure and monitor the quality of data in data lakes and pipelines. It allows users to define rules, evaluate them against datasets, and take action on records that fail quality checks, directly addressing the requirements of the question.
• C. Incorrect. AWS Glue Triggers are used for orchestrating and automating ETL workflows. They can start jobs or crawlers based on a schedule or events, but they do not have any capability to define or enforce data quality rules themselves.
• D. Correct. The Data Quality Definition Language (DQDL) is a domain-specific language used within the AWS Glue Data Quality feature to declaratively define the rules. For example, a rule like `IsComplete "product_id"` would check for the column's existence, and `ColumnValues "sale_price" between 0.00 and 9999.99` would check the value range, fulfilling the scenario's requirements.
• E. Incorrect. While you can use the AWS Glue Studio visual editor to build a job that includes a data quality transformation node, AWS Glue Studio is the integrated development environment (IDE), not the underlying feature itself. The core feature is AWS Glue Data Quality (B), and the language used is DQDL (D).

Correct answer: D — Completeness

• A. Incorrect. Validity refers to whether the data conforms to defined formats, structures, or business rules (e.g., a phone number must have a certain number of digits). Missing data is not an issue of invalid formatting.
• B. Incorrect. Consistency ensures that data is uniform and does not contradict itself across different systems or datasets. For example, a customer's name should be consistent in both the sales and support databases. A gap in data for a time period is not a contradiction.
• C. Incorrect. Accuracy measures how well the data reflects the true, real-world values or events it is meant to represent. The issue here is the absence of data, not the presence of incorrect or erroneous data values.
• D. Correct. Completeness is the dimension of data quality that measures whether all expected data is present. The report of missing sensor data from a specific time interval is a classic example of an incomplete dataset, indicating a failure in this dimension.

Correct answer: B — Use an AWS CloudFormation template to define and deploy both the Glue job and its associated CloudWatch alarm.

• A. This approach is incorrect because manually creating alarms is inefficient, not scalable, and highly prone to human error. It directly contradicts the requirement for an automated and standardized solution.
• B. This is the correct and most efficient solution. Using AWS CloudFormation enables the practice of Infrastructure as Code (IaC), allowing you to define the AWS Glue job, the CloudWatch alarm, and the SNS notification topic within a single template. This ensures that every new job is deployed with a standardized, consistent monitoring configuration, achieving robust automation and reducing both manual effort and the risk of configuration drift.
• C. While using an AWS CLI script within a CI/CD pipeline can achieve automation, it is not the most efficient or maintainable method compared to CloudFormation. Managing imperative scripts adds operational overhead, whereas CloudFormation provides a declarative, version-controlled, and integrated way to manage related AWS resources as a single, atomic unit (a stack).
• D. This is incorrect. AWS Config is a service for assessing, auditing, and evaluating the configurations of AWS resources for compliance. Its primary function is not to provision new resources like CloudWatch alarms. While it can trigger remediation actions, using it for this purpose would be an unconventional and inefficient implementation compared to dedicated IaC tools like CloudFormation.

Correct answer: C — Glue Interactive Sessions

• A. Incorrect. Glue Job Bookmarks are used to track data that has already been processed, preventing the reprocessing of the same data in subsequent job runs. They are essential for incremental data processing but do not provide any capabilities for interactive script development or debugging.
• B. Incorrect. Glue Workflows are used to create, visualize, and manage complex ETL pipelines by orchestrating multiple Glue jobs, crawlers, and triggers. They are for managing the overall flow of data processing, not for the interactive development or debugging of an individual script's logic.
• C. Correct. Glue Interactive Sessions provide a serverless, on-demand Spark environment that you can connect to from a Jupyter notebook or IDE. This allows a data engineer to interactively run, test, and debug their ETL code line by line against a specific dataset, such as the problematic monthly file, making it the ideal tool for this scenario.
• D. Incorrect. Glue Data Quality rules allow you to define rules to evaluate the quality of your data within a pipeline. While they can help identify that malformed records exist, they do not provide the interactive environment needed to develop and debug the Spark transformation logic required to handle or fix those records.

Correct answer: C — By configuring the script as a bootstrap action for the EMR cluster.

• A. Incorrect. EMR steps are executed after the cluster is fully provisioned and in a 'WAITING' state. While a script can be run as a step, it does not meet the requirement of running immediately after node provisioning and before any processing steps begin. Bootstrap actions are the appropriate mechanism for this initial setup.
• B. Incorrect. A Lambda function is external to the EMR cluster. While it can be triggered by a cluster creation event, it does not have direct access to the cluster's nodes to run scripts. Implementing this would be overly complex and less secure compared to using the native EMR feature designed for this task.
• C. Correct. Bootstrap actions are the standard and recommended AWS mechanism for this use case. They are scripts that Amazon EMR runs on every node in the cluster right after the node is provisioned but before applications like Spark or Hadoop are started and before any steps are executed. This is the ideal lifecycle hook for installing custom software, libraries, or configuring environment variables.
• D. Incorrect. This is a manual, non-scalable, and error-prone process that goes against automation best practices. It would also only configure the master node by default, not all nodes in the cluster as required.

Correct answer: C — As arguments (e.g., `retries`, `retry_delay`) within the operator definition in the Airflow DAG python script.

• A. Incorrect. While AWS Step Functions has its own retry capabilities, the question states the pipeline is orchestrated by Amazon MWAA (Apache Airflow). The control of task-level execution, including retries for transient errors, is managed within the Airflow DAG itself, not by an external orchestration service like Step Functions.
• B. Incorrect. AWS IAM (Identity and Access Management) roles are used to manage security permissions, defining what actions the MWAA environment can perform on other AWS resources. IAM has no functionality for configuring operational logic such as task retries or delays.
• C. Correct. Apache Airflow (the engine for MWAA) is designed to handle task failures and retries natively. The standard and correct way to implement this is by setting parameters like `retries` and `retry_delay` directly in the DAG's Python script. These can be applied to all tasks via `default_args` or overridden for specific tasks within their operator definitions.
• D. Incorrect. Amazon EventBridge is a serverless event bus that can be used to trigger DAG runs on a schedule or in response to events. Its role ends once it initiates the DAG run. It does not control the internal execution, error handling, or retry logic of individual tasks within the running DAG.

Correct answer: B — Using the 'Consolidate' or 'Map' transformation to group multiple values into a single value.

• A. Incorrect. One-hot encoding is a technique used in machine learning to convert categorical variables into a binary vector format. It does not standardize or consolidate different text values into a single category; instead, it creates new columns for each category.
• B. Correct. AWS Glue DataBrew provides transformations specifically for this purpose. The 'Map values' transformation allows you to define explicit rules to group multiple distinct source values (e.g., 'U.S.A.', 'United States', 'America') into a single, standardized target value ('USA'). This is the most direct and efficient method to solve the problem.
• C. Incorrect. Splitting a column by a delimiter is used when a single cell contains multiple pieces of information separated by a character, like a comma or a hyphen. This is not applicable for standardizing single, varied text entries like country names.
• D. Incorrect. Filtering would remove rows that don't match the criteria (e.g., rows with 'America'), resulting in significant data loss. The objective is to clean and standardize the existing data, not to delete it.

Correct answer: C — Substituting missing values with estimated or calculated values.

• A. Incorrect. This process is generally referred to as data filtering or cleaning by removal. While it is a data quality technique, it is distinct from imputation, which aims to preserve records by filling in missing data rather than deleting the records entirely.
• B. Incorrect. This process is known as data standardization or normalization. It involves transforming data into a consistent and common format (e.g., converting all date strings to 'YYYY-MM-DD'), but it does not address missing values.
• C. Correct. Imputation is a statistical technique used to handle missing data by replacing null or missing values with substituted, estimated values. Common methods include replacing missing values with the mean, median, mode of a column, or using more complex predictive models.
• D. Incorrect. This process is called deduplication. It involves identifying and removing duplicate records from a dataset to ensure each entry is unique and to maintain data integrity. It does not involve handling missing values within a record.

Correct answer: B — To automatically discover, classify, and protect sensitive data at scale.

• A. This is incorrect. Amazon Macie is a data security and privacy service, not a firewall. Firewall services in AWS that protect resources like S3 are provided by services such as AWS WAF and AWS Network Firewall, which control access based on rules, whereas Macie inspects the data itself.
• B. This is the correct primary function of Amazon Macie. It is a fully managed service that uses machine learning and pattern matching to discover, classify, and help you protect sensitive data, such as personally identifiable information (PII) and intellectual property, stored in Amazon S3 at scale.
• C. This is incorrect. The primary service for monitoring and logging API calls and user activity across an AWS account is AWS CloudTrail. While Macie can provide insights related to data access, its core function is data discovery and classification, not comprehensive activity monitoring.
• D. This is incorrect. Managing fine-grained, column-level access permissions for databases and tables within a data lake is the primary function of AWS Lake Formation, which builds upon AWS IAM policies. Amazon Macie does not manage access permissions but rather identifies what data needs to be protected.

Correct answers: A, C — Attach an IAM policy to user roles that explicitly denies the `rds:CopyDBSnapshot` action for non-North American regions.; Create a Service Control Policy (SCP) that denies the `rds:CopyDBSnapshot` action if the target region is outside North America.

• A. This is a correct and effective preventive measure. By attaching an IAM policy with an explicit `Deny` statement for the `rds:CopyDBSnapshot` action to all relevant user roles, you can enforce the data residency policy at the identity level. Using condition keys like `aws:RequestedRegion` in the policy allows you to specify that the denial only applies when the target region is outside of the approved list (e.g., North American regions).
• B. This is not a sufficient preventive control. While you would configure an AWS Backup plan to not copy snapshots to non-compliant regions as part of your backup strategy, this does not prevent a user or role with the necessary permissions from manually initiating a snapshot copy via the console or API. This option addresses the configuration of the automated process but fails to act as a security guardrail against ad-hoc actions.
• C. This is a correct and highly recommended preventive measure. Service Control Policies (SCPs) are a feature of AWS Organizations that apply guardrails to all principals (including the root user) within an account or Organizational Unit (OU). An SCP that denies the `rds:CopyDBSnapshot` action for non-compliant regions provides the broadest and most robust enforcement of the data residency policy, overriding any conflicting IAM permissions.
• D. This is incorrect. Security groups function as a stateful firewall for resources like EC2 and RDS instances, controlling inbound and outbound network traffic at the instance level. The action of copying an RDS snapshot is a control plane API call managed by the AWS backbone, not network traffic originating from the RDS instance itself. Therefore, a security group cannot block this action.
• E. This is incorrect because it describes a reactive, not a preventive, control. The question requires preventing the copy from happening. With AWS Config, the non-compliant copy would first be created, leading to a temporary violation of the data residency policy. Only after this violation is detected would a remediation action (like deletion) be triggered. This 'detect and remediate' approach does not meet the requirement to prevent the action.

Correct answer: C — Amazon EMR

• A. Incorrect. AWS Lambda is a serverless compute service for short-running, event-driven functions. It has strict limitations on execution duration (maximum 15 minutes) and available memory/storage, making it completely unsuitable for processing a 5 TB daily batch job with complex Spark logic.
• B. Incorrect. While AWS Glue is a serverless ETL service that can run Apache Spark jobs, it is generally optimized for more standard ETL workflows. For a very large-scale (5 TB daily) and complex job requiring significant customization, performance tuning, and granular control over the Spark environment, AWS Glue may not be as performant or cost-effective as Amazon EMR. EMR offers more flexibility and control over the underlying cluster.
• C. Correct. Amazon EMR is a managed big data platform specifically designed to run large-scale distributed data processing frameworks like Apache Spark. It provides the required scalability, flexibility, and performance to handle a 5 TB daily batch job. EMR allows for deep customization of the Spark environment and cluster hardware, which is essential for optimizing performance and cost on such a large and complex workload.
• D. Incorrect. Amazon Kinesis Data Analytics is a service for real-time processing of streaming data. The requirement is for a daily batch processing job on data stored in S3, not for processing a continuous stream of data. Therefore, this service is not a suitable choice for the described use case.

Correct answers: A, C — A CloudWatch Logs subscription filter; Amazon Kinesis Data Firehose

• A. Correct. A CloudWatch Logs subscription filter is the standard mechanism to access a real-time feed of log events from a log group. It can be configured to forward matching log events to a destination like Amazon Kinesis Data Firehose, AWS Lambda, or Amazon Kinesis Data Streams, initiating the streaming pipeline directly from the source.
• B. Incorrect. Amazon S3 event notifications are used to trigger actions in response to events occurring within an S3 bucket, such as object creation or deletion. They are not used for streaming data from CloudWatch Logs.
• C. Correct. Amazon Kinesis Data Firehose is a fully managed service for delivering real-time streaming data to various destinations, including Amazon OpenSearch Service. It can be set as the destination for a CloudWatch Logs subscription filter, creating a direct, serverless pipeline that requires no custom code for ingestion and delivery.
• D. Incorrect. AWS Data Pipeline is a service for orchestrating scheduled, batch-based data processing and movement workflows. It is not designed for continuous, real-time data streaming and is therefore unsuitable for this use case.
• E. Incorrect. AWS Glue is a serverless ETL service primarily used for data discovery, preparation, and integration, typically in batch or micro-batch scenarios. While Glue has streaming capabilities, using Kinesis Data Firehose is a much simpler, more direct, and more appropriate solution for streaming logs to OpenSearch with minimal code.

Correct answers: A, C — In Account A, grant permissions on the table to the Account B ID.; In Account B, the data engineer must create a resource link pointing to the shared table.

• A. This is a mandatory first step in the data owner account (Account A). The data governance team must explicitly grant AWS Lake Formation permissions (e.g., SELECT) on the specific table to the consumer's AWS Account ID (Account B). This action makes the table available to be shared and discoverable by the consumer account.
• B. This is incorrect. The Lake Formation cross-account sharing model does not require the Lake Formation service in the owner account (A) to assume a role in the consumer account (B). Instead, the owner account grants permissions directly to the consumer account's ID. The consumer account administrator then grants permissions on the shared resource to their local IAM principals.
• C. This is a required step in the data consumer account (Account B). After an administrator in Account B accepts the shared table resource, a resource link must be created in Account B's AWS Glue Data Catalog. This resource link acts as a local pointer or alias to the shared table in Account A, enabling services like Amazon Athena in Account B to query it.
• D. This is incorrect and represents a major security anti-pattern. Permissions should never be granted to the root user. While the S3 bucket policy in Account A must ultimately permit access from the querying principal in Account B, Lake Formation is the primary control plane for managing these permissions, not direct, overly permissive S3 bucket policies.
• E. This is incorrect. A key benefit of AWS Lake Formation's cross-account feature is to enable a centralized data catalog without the need for replication. Creating a replica of the catalog would introduce data silos and management overhead, which is precisely what Lake Formation aims to prevent. The correct mechanism is to use resource links to reference the shared objects directly.

Correct answer: B — To securely store, manage, and rotate credentials, API keys, and other secrets.

• A. Incorrect. This describes the functionality of Amazon Macie, a data security service that uses machine learning to discover and protect sensitive data in AWS, such as personally identifiable information (PII) within Amazon S3 buckets.
• B. Correct. The primary function of AWS Secrets Manager is to manage the lifecycle of secrets, including credentials, API keys, and tokens. In a data engineering pipeline, this is crucial for securely providing applications and services with the credentials they need to access databases, data warehouses, and other resources without hardcoding them.
• C. Incorrect. This is the primary function of AWS CloudTrail, which provides event history of your AWS account activity, including actions taken through the AWS Management Console, AWS SDKs, command line tools, and other AWS services. It is used for governance, compliance, operational auditing, and risk auditing.
• D. Incorrect. This functionality is primarily handled by AWS Key Management Service (KMS), which allows you to create and manage cryptographic keys. While Secrets Manager uses KMS to encrypt the secrets it stores, its main function is secret management, not the direct encryption of data within services like S3 or Redshift.

Correct answers: A, E — Server-Side Encryption with Amazon S3-Managed Keys (SSE-S3); Server-Side Encryption with Customer-Provided Keys (SSE-C)

• A. Correct. Server-Side Encryption with S3-Managed Keys (SSE-S3) is one of the three primary server-side encryption methods for S3. With SSE-S3, Amazon S3 manages both the data key and the master key, providing a simple, hands-off encryption solution.
• B. Incorrect. This is not a recognized AWS S3 encryption option. The term appears to be a fabricated combination of 'Server-Side Encryption with Customer-Provided Keys' (SSE-C) and terminology related to client-side encryption or KMS keys.
• C. Incorrect. While you can use Server-Side Encryption with AWS KMS (SSE-KMS) and have the KMS key backed by a custom key store using AWS CloudHSM, this describes a specific implementation detail of SSE-KMS rather than a distinct, primary encryption option. SSE-S3 and SSE-C represent two of the three fundamental and distinct server-side encryption types, making them better choices.
• D. Incorrect. This option describes client-side encryption, where data is encrypted by the client *before* being uploaded to Amazon S3. The question specifically asks for server-side encryption options, where S3 performs the encryption upon receiving the object.
• E. Correct. Server-Side Encryption with Customer-Provided Keys (SSE-C) is one of the three primary server-side encryption methods. With this option, the customer provides their own encryption key along with each request to S3. S3 uses the key to perform encryption or decryption and then immediately discards the key, never storing it.

Correct answers: C, D — Use the AWS Transfer Family with SFTP, FTPS, or FTP.; Use HTTPS endpoints when uploading files via the AWS CLI or SDKs.

• A. Incorrect. AWS Direct Connect establishes a private, dedicated network connection between an on-premises data center and AWS. It does not operate over the public internet, which contradicts the requirements of the question. Furthermore, Direct Connect does not provide encryption in transit by default; a separate solution like a VPN or MACsec would be required.
• B. Incorrect. While AWS Storage Gateway does transfer data to AWS over HTTPS, thereby encrypting it in transit, it is a higher-level hybrid cloud storage service rather than a direct transfer mechanism. Options like using HTTPS endpoints or SFTP/FTPS are more fundamental and direct answers to the question about encryption mechanisms.
• C. Correct. AWS Transfer Family is a fully managed service that supports secure file transfer protocols like SFTP (SSH File Transfer Protocol) and FTPS (File Transfer Protocol over SSL/TLS). These protocols are specifically designed to encrypt data and commands during transit, providing a secure method for transferring files to Amazon S3 over the public internet.
• D. Correct. This is a standard and highly recommended method for securing data in transit to Amazon S3. By default, the AWS CLI, AWS SDKs, and the S3 console all use HTTPS endpoints. HTTPS leverages the Transport Layer Security (TLS) protocol to create an encrypted channel, ensuring that all data transferred between the client and S3 is protected from eavesdropping or tampering over the public internet.
• E. Incorrect. Amazon S3 Transfer Acceleration is a feature designed to improve the performance and speed of file transfers over long distances by routing traffic through AWS's optimized global network edge locations. It does not provide encryption itself. Encryption is handled by the underlying protocol used for the transfer, which is typically HTTPS. Therefore, it is a performance feature, not a security mechanism.

Correct answer: B — To act as a stateful virtual firewall for an instance, controlling inbound and outbound traffic.

• A. Incorrect. This statement describes a Network Access Control List (NACL), not a security group. Security groups are stateful, meaning return traffic is automatically allowed, and they operate at the instance level. NACLs are stateless and operate at the subnet level.
• B. Correct. A security group acts as a stateful virtual firewall for associated resources like EC2 instances. It controls both inbound and outbound traffic based on a set of 'allow' rules. Because it's stateful, if you allow inbound traffic on a specific port, the corresponding outbound return traffic is automatically permitted, regardless of outbound rules.
• C. Incorrect. The function of a security group is to control network traffic access, not to encrypt it. Network traffic encryption must be handled by other mechanisms, such as configuring applications to use protocols like TLS/SSL.
• D. Incorrect. Security groups do not have explicit 'deny' rules. They operate on an 'allow-only' principle, where any traffic not explicitly permitted by an 'allow' rule is implicitly denied. Network ACLs, in contrast, support both explicit 'allow' and 'deny' rules.

Correct answer: A — Do not attach an IAM role to the EC2 instance. Require each user to run `aws configure` with their own IAM user access keys upon their first login.

• A. Correct. This solution leverages the AWS CLI credential provider chain. By not attaching an instance role, the lowest-priority credential source is removed. When users run `aws configure`, they create a credentials file (`~/.aws/credentials`) in their unique home directory. The AWS CLI will find and use these user-specific credentials by default, ensuring that all commands are executed with their individual IAM user permissions.
• B. Incorrect. Attaching a `DenyAll` role is an ineffective approach. The EC2 instance metadata service would still provide credentials from this role, and the CLI would attempt to use them first if user credentials are not explicitly configured. While any API call would fail due to the `DenyAll` policy, this approach does not enforce the use of individual IAM user credentials and is a confusing, non-standard configuration.
• C. Incorrect. While AWS Systems Manager Session Manager provides secure access to an instance, it does not change how the AWS CLI resolves credentials for commands run within the session. The CLI would still follow the standard credential provider chain and default to using the instance role's permissions if a role is attached, which is the opposite of the stated requirement.
• D. Incorrect. This statement is false. The AWS credential provider chain has a well-defined order of precedence. Credentials specified in environment variables or in the shared credentials file (`~/.aws/credentials`) are checked *before* the EC2 instance role. Therefore, explicitly configured user credentials will always take precedence over the instance role.