Free Confluent Certified Developer for Apache Kafka® Sample Questions

Correct answer: A — Key, Value, Timestamp, and Headers

• A. Correct. A standard Kafka record (message) consists of a Key, Value, Timestamp, and optional Headers. These components are the fundamental parts exposed by Kafka producer/consumer APIs and defined in the Kafka wire protocol.
• B. Incorrect. While Key and Value are core components, the Schema ID is specific to the Schema Registry and serialization formats (like Avro), not the Kafka record structure itself. The Offset is metadata assigned by the broker once the message is appended to the log.
• C. Incorrect. Topic and Partition are metadata fields that specify where a record is stored, but they are not internal components of the record payload. The CRC checksum is used for internal data integrity checking rather than being a functional component of the message record.
• D. Incorrect. Kafka messages do not use a 'Footer' structure. The standard components are key, value, timestamp, and headers; generic 'metadata' is handled through specific fields like headers or broker-assigned properties (offset, partition).

Correct answer: A — A list of replicas that are fully caught up with the partition leader.

• A. Correct. The In-Sync Replica (ISR) list consists of replicas that are fully caught up with the partition leader. These replicas have replicated all messages that the leader has acknowledged (within the configured 'replica.lag.time.max.ms'). Members of the ISR are eligible to be elected as the new leader if the current leader fails, ensuring durability and high availability.
• B. Incorrect. Offline or unreachable replicas are excluded from the ISR. Replicas that lag significantly or become unavailable are removed from the ISR until they catch up with the leader and satisfy the synchronization requirements again.
• C. Incorrect. The ISR tracks replica synchronization status for specific partitions, not assignment to the controller. While the controller manages cluster metadata and updates the ISR in ZooKeeper or KRaft, the ISR list itself describes the state of replicas for a partition across various brokers.
• D. Incorrect. The ISR list's primary purpose is to maintain write durability (via the 'acks=all' setting) and to provide a pool of candidates for leader election. While Kafka supports fetching from followers (Rack Awareness), the ISR list is not defined as a set of replicas that only serve read requests.

Correct answer: A — `processing.guarantee="exactly_once_v2"`

• A. Correct. In Kafka Streams, the `processing.guarantee` configuration is the primary setting used to enable exactly-once semantics. The value `exactly_once_v2` (introduced in Kafka 2.5) is the current recommended setting, offering better scalability and performance than the original `exactly_once` by using a more efficient transactional model. This setting automatically handles the necessary internal configurations for producers and consumers.
• B. Incorrect. While `enable.idempotence=true` is a producer-level configuration that prevents duplicate messages from being written to the log, it is not the high-level configuration used to enable EOS for a Streams application. Kafka Streams manages producer settings automatically when EOS is enabled via `processing.guarantee`.
• C. Incorrect. `isolation.level="read_committed"` is a consumer-level configuration that ensures only records from successfully committed transactions are visible. While Kafka Streams configures this internally when EOS is on, manually setting it does not enable end-to-end exactly-once semantics.
• D. Incorrect. `acks="all"` is a producer setting used to ensure that messages are acknowledged by all in-sync replicas for maximum durability. While required for data integrity within EOS, it is a durability setting rather than the Kafka Streams-specific EOS toggle.

Correct answers: A, B — Performing partition leader elections; Managing topic creation and deletion

• A. Correct. The active Kafka Controller is responsible for coordinating partition leader elections. This includes handling broker failures by electing new leaders for affected partitions and managing preferred leader elections to maintain cluster balance.
• B. Correct. Managing topic creation and deletion is a primary responsibility of the active Kafka Controller. It handles the administrative metadata changes and ensures that partition and replica states are updated across all brokers in the cluster.
• C. Incorrect. Consumer group offsets are stored in the internal __consumer_offsets topic and are managed by the Group Coordinator (a role assigned to a broker), not specifically by the cluster's active Controller.
• D. Incorrect. Log compaction is a function of the Log Cleaner threads, which run on every broker to manage local log segments. It is not a centralized task managed by the Controller.
• E. Incorrect. Routing producer requests is handled by the Kafka clients themselves. Clients use metadata to determine which broker is the leader for a specific partition and send requests directly to that broker; the Controller is not involved in the data path.

Correct answers: A, B, C — A new consumer instance joins the consumer group.; An existing consumer instance crashes or leaves the group.; New partitions are added to a topic the group is subscribed to.

• A. Correct. When a new consumer instance joins an existing consumer group, the group membership changes and Kafka must redistribute partitions among the active members to ensure the load is shared.
• B. Correct. If a consumer crashes (stops sending heartbeats) or voluntarily leaves the group, the Group Coordinator triggers a rebalance to reassign the orphaned partitions to the remaining active members.
• C. Correct. Adding new partitions to a subscribed topic changes the set of partitions that the group must manage. A rebalance is required to assign these new partitions to the available consumers.
• D. Incorrect. Adding a broker to the cluster increases capacity but does not directly affect consumer group membership or the number of partitions in a topic, so it does not trigger a rebalance.
• E. Incorrect. Tombstone messages are used for record deletion in compacted topics; they affect data storage and retention but have no impact on consumer group partition assignments.
• F. Incorrect. A controller election changes how cluster-wide metadata and partition leadership are managed, but it does not directly trigger consumer group rebalancing, which is handled by the Group Coordinator.

Correct answers: A, B — `replication.factor=3`; `min.insync.replicas=2`

• A. Setting `replication.factor=3` ensures that each partition has a copy on every broker in a 3-broker cluster. This provides the redundancy required for high availability and durability, as two replicas will still exist if one broker fails.
• B. Setting `min.insync.replicas=2` (typically used with producer `acks=all`) ensures that a write is only successful if at least two replicas acknowledge it. In a 3-broker cluster with a replication factor of 3, this allows the topic to remain available for writes if one broker goes down (since 2 replicas remain), while still guaranteeing that data is written to multiple brokers for durability.
• C. The number of partitions controls parallelism and throughput. While a single partition is valid, it does not provide high availability or durability guarantees in the event of a broker failure.
• D. `cleanup.policy=compact` triggers log compaction, which ensures Kafka retains the latest value for a specific key. This is a retention strategy and does not manage replication or availability during broker outages.
• E. `retention.ms=-1` specifies that data should be kept indefinitely. While this preserves data over time, it does not ensure that the data is replicated across brokers or that the topic remains writable during a broker failure.

Correct answer: B — 1

• A. Setting acks to '0' means the producer will not wait for any acknowledgment from the server at all. The record is considered sent as soon as it is written to the network buffer, providing the highest throughput but the lowest reliability.
• B. Setting acks to '1' means the leader will write the record to its local log and then respond to the producer without waiting for full replication from all followers. This provides a balance between latency and durability.
• C. Setting acks to 'all' (or '-1') means the leader will wait for the full set of in-sync replicas (ISRs) to acknowledge the record. This ensures the record will not be lost as long as at least one in-sync replica remains alive.
• D. There is no 'none' value for the 'acks' configuration in Apache Kafka. To specify that no acknowledgment is required, the value '0' must be used.

Correct answer: B — Hopping Windows

• A. Tumbling windows are fixed-size, non-overlapping, and contiguous time intervals. Each record belongs to exactly one window because the window size is equal to the advance interval.
• B. Hopping windows are defined by two properties: the window size and the advance (hop) interval. If the advance interval is smaller than the window size, the windows overlap, allowing a single record to be contained in multiple windows.
• C. Sliding windows are used primarily for join operations and do not have a fixed advance interval. Instead, windows are defined by the maximum time difference between two records.
• D. Session windows are not fixed-size; they are based on periods of activity and are separated by a specified gap of inactivity. They do not overlap by definition, as any overlapping activity would be merged into the same session.

Correct answer: C — CooperativeStickyAssignor

• A. Incorrect. The RangeAssignor is the default strategy and assigns partitions on a per-topic basis. It does not prioritize minimizing partition movement and can often lead to an uneven distribution of partitions across consumers when multiple topics are involved.
• B. Incorrect. While the RoundRobinAssignor distributes partitions of all subscribed topics across consumers in a sequential fashion to ensure a balanced load, it does not minimize partition movements during rebalances and relies on the 'stop-the-world' eager rebalance protocol.
• C. Correct. The CooperativeStickyAssignor (introduced in Kafka 2.4) follows the same logic as the StickyAssignor to minimize partition movement but utilizes the incremental cooperative rebalance protocol (KIP-429). This allows consumers to continue processing data from their assigned partitions during a rebalance, significantly reducing group downtime.
• D. Incorrect. The StickyAssignor aims to balance partitions while maximizing 'stickiness' (keeping existing assignments stable). However, it uses the eager rebalance protocol, which requires all consumers to revoke their partitions before any reassignments can occur, making it less efficient than the CooperativeStickyAssignor.

Correct answer: B — `retries=0`

• A. Incorrect. The `acks=1` setting ensures the leader replica has received the record but does not disable retries. If a network error occurs during acknowledgment, the producer may retry, resulting in duplicate records if the leader had actually succeeded in the previous attempt.
• B. Correct. Setting `retries=0` prevents the producer from attempting to resend messages after a failure or timeout. While this eliminates the possibility of duplicate records being created due to retries, it increases the risk of data loss because messages that fail their first transmission attempt are not resent.
• C. Incorrect. `max.in.flight.requests.per.connection <= 5` is a specific requirement for the idempotent producer (Kafka 1.1+) to ensure ordering and handle duplicates correctly. However, on its own (with idempotence disabled), it does not prevent the producer from retrying and creating duplicates.
• D. Incorrect. `isolation.level=read_committed` is a consumer-side configuration used in conjunction with Kafka Transactions. It ensures consumers only read messages from successfully committed transactions and has no impact on producer-side retries or duplicate prevention.

Correct answers: A, B, C — Increase `linger.ms`; Increase `batch.size`; Enable `compression.type`

• A. Increasing `linger.ms` tells the producer to wait up to that many milliseconds for more records to arrive before sending the batch. This results in larger batches and better throughput by reducing the total number of requests sent to the broker, though it introduces a small amount of latency.
• B. Increasing `batch.size` raises the upper limit of the amount of data (in bytes) that the producer will bundle into a single batch per partition. Larger batches allow for more efficient processing and better compression ratios, directly increasing throughput.
• C. Enabling `compression.type` (e.g., snappy, lz4, or zstd) reduces the size of the data sent over the network and stored on disk. By reducing the network bandwidth bottleneck and the overhead per record, it effectively increases the throughput of the producer.
• D. Setting `acks=all` ensures the highest durability by requiring acknowledgments from all in-sync replicas (ISRs). This increases latency and reduces throughput compared to `acks=1` or `acks=0`.
• E. Decreasing `max.request.size` restricts the maximum size of a produce request. This would likely decrease throughput because the producer would need to send more individual requests to transfer the same amount of data.
• F. Setting `max.in.flight.requests.per.connection=1` ensures message ordering by allowing only one unacknowledged request at a time. This severely limits throughput because it prevents the producer from utilizing concurrent network requests.

Correct answer: B — KTable

• A. A KStream represents a record stream where each record is an independent event in an insert-only fashion. It does not treat records as updates to previous values for the same key.
• B. A KTable is an abstraction of a changelog stream where each record is interpreted as an update to the previous value associated with that key. If a record with a null value is received, it represents a deletion (tombstone).
• C. A GlobalKTable is a table that is fully replicated across all instances of a Kafka Streams application. While it shares the update semantics of a KTable, the primary and fundamental abstraction for an update stream is the KTable.
• D. A KGroupedStream is an intermediate abstraction used after a KStream has been grouped by key. It is used as a prerequisite for aggregations (like count or reduce) and is not the primary abstraction for representing a persistent update stream.

Correct answer: B — A `TopologyBuilderException` will be thrown at runtime due to a partition mismatch.

• A. Incorrect. Kafka Streams requires that the topics involved in a join operation be co-partitioned. This means they must have the same number of partitions to ensure that records with the same key are processed by the same stream task.
• B. Correct. Kafka Streams enforces co-partitioning requirements (same number of partitions) for joins between a KStream and a KTable. If the partition counts do not match (6 vs 4), the application will fail to start and throw a `TopologyBuilderException` (or a similar validation error) because it cannot guarantee that keys will align across the stream tasks.
• C. Incorrect. Kafka Streams does not automatically repartition source topics to resolve a mismatch in partition counts. While you can manually use the `.repartition()` operator in the DSL, it does not happen implicitly for the source topics of a join.
• D. Incorrect. Kafka Streams does not silently drop data or ignore partitions to compensate for a configuration error. It enforces strict compatibility rules to ensure data integrity and fails fast during the topology initialization phase.

Correct answer: B — The maximum number of partitions across all input topics.

• A. The num.stream.threads setting controls how many processing threads a single Kafka Streams instance can use to execute its assigned tasks, but it does not determine the total number of tasks created for the application.
• B. Kafka Streams creates tasks based on the partitions of the input topics. The total number of stream tasks is determined by the maximum number of partitions among all input topics of the topology. This ensures that every partition can be processed in parallel by a specific task.
• C. State stores are used for storing and querying local state (e.g., in joins or aggregations) within the topology. While tasks may use state stores, the number of tasks is driven by input partitions, not the number of stores.
• D. The number of application instances determines how tasks are distributed across different machines or processes for horizontal scaling, but the total number of tasks itself is pre-determined by the source topic partitions.

Correct answer: A — Set `default.deserialization.exception.handler` to `LogAndContinueExceptionHandler`.

• A. Correct. The `LogAndContinueExceptionHandler` is specifically designed for this scenario. It logs the deserialization error and skips the problematic record (the 'poison pill'), allowing the Kafka Streams application to continue processing subsequent records without shutting down.
• B. Incorrect. `LogAndFailExceptionHandler` is the default behavior in Kafka Streams. It logs the error but then fails the stream thread, which stops processing. This does not meet the requirement to drop the record and continue.
• C. Incorrect. In Kafka Streams, the consumer poll loop is managed internally by the framework and is abstracted away from the developer. You do not have direct access to wrap the poll loop in a try-catch block to handle deserialization errors occurring within the internal StreamThread.
• D. Incorrect. While a Dead Letter Queue (DLQ) is a valid pattern for handling bad data, Kafka Streams does not provide a built-in DLQ configuration directly within `StreamsBuilder` for deserialization exceptions. To implement a DLQ for deserialization errors, you would typically need to write a custom exception handler that implements the `DeserializationExceptionHandler` interface.

Correct answers: B, C — It processes records synchronously without using actual threads.; It allows querying state stores directly to verify intermediate state.

• A. Incorrect. TopologyTestDriver is a unit testing utility that simulates the Kafka environment in-memory. It does not require a running Kafka broker or infrastructure tools like Testcontainers.
• B. Correct. Records piped into the TopologyTestDriver are processed synchronously on the calling thread. This ensures deterministic behavior and eliminates the complexity of managing background threads during testing.
• C. Correct. The TopologyTestDriver provides a method to access and query the state stores (via `getStateStore`) defined in the topology, allowing for easy verification of the processing logic's side effects.
• D. Incorrect. Since it is a synchronous, in-memory tool for testing business logic, it does not simulate distributed system failures like network partitions or broker outages.
• E. Incorrect. While you can test topologies that use Avro, you do not need a live Schema Registry. Developers typically use a MockSchemaRegistryClient to handle schema-based SerDes within the test environment.

Correct answers: A, C, F — It is fully replicated to every Kafka Streams instance.; It allows joining with a KStream using a non-key attribute.; It is populated by reading all partitions of the underlying topic.

• A. Correct. A GlobalKTable is replicated in full to every Kafka Streams instance, ensuring each instance has the complete dataset locally. This distinguishes it from a regular KTable, which is partitioned across instances.
• B. Incorrect. One of the primary advantages of a GlobalKTable is that it does not require co-partitioning for joins with a KStream, as the full table is available on every instance.
• C. Correct. A GlobalKTable join allows the KStream to map a non-key attribute to the table's key via a KeyValueMapper, enabling lookups based on values within the stream records rather than just the record key.
• D. Incorrect. While GlobalKTables use state stores, they are primarily defined by their replication and partition consumption model. They typically use the source topic directly for restoration across all instances.
• E. Incorrect. While it avoids the network shuffle of the KStream during a join, a GlobalKTable often increases overall network and storage overhead because the entire topic is consumed and stored by every single application instance.
• F. Correct. To maintain a full copy of the data on every instance, Kafka Streams assigns all partitions of the underlying input topic to every instance of the application.

Correct answer: B — Modifying messages on the fly as they flow through Kafka Connect.

• A. Incorrect. SMTs are stateless and operate on individual records one at a time. Aggregating multiple messages requires stateful processing, which is better suited for Kafka Streams or ksqlDB. While Kafka Connect supports internal batching for transport efficiency, this is handled by the framework, not SMTs.
• B. Correct. Single Message Transforms (SMTs) are used for lightweight, on-the-fly transformations of individual records as they pass through a Source or Sink connector. Common use cases include renaming fields, masking PII, routing records based on content, or adding metadata headers.
• C. Incorrect. Format conversion (serialization/deserialization) is the specific responsibility of Converters (e.g., JsonConverter, AvroConverter). SMTs operate on the internal Kafka Connect data format after a source record has been parsed or before a sink record is serialized.
• D. Incorrect. Managing offsets is a core responsibility of the Kafka Connect runtime and the underlying Kafka consumer/producer clients to ensure reliability and fault tolerance. It is not a function of the transformation layer.

Correct answer: A — GET /connectors/{name}/status

• A. Correct. The GET /connectors/{name}/status endpoint is the standard and correct way to retrieve the current operational state of a connector (such as RUNNING, FAILED, or PAUSED) and its tasks, including worker assignments.
• B. Incorrect. While 'state' is a descriptive term for what is being retrieved, the actual REST API path defined in Apache Kafka documentation is /status, not /state.
• C. Incorrect. This is not a valid endpoint in the Kafka Connect REST API. Status retrieval is performed via a GET request, whereas POST requests are used for actions like restarting a connector.
• D. Incorrect. All connector-specific endpoints in the Kafka Connect REST API are structured under the /connectors base path (e.g., /connectors/{name}/status).

Correct answer: A — connect-configs, connect-offsets, connect-status

• A. Correct. Kafka Connect in distributed mode requires three internal topics to manage its state: one for connector and task configurations (config.storage.topic), one for source connector offsets (offset.storage.topic), and one for the current status of connectors and tasks (status.storage.topic). By convention, these are often named connect-configs, connect-offsets, and connect-status.
• B. Incorrect. __consumer_offsets is used by the Kafka broker to store offsets for all consumer groups, and __transaction_state stores metadata for transactional producers. While Kafka Connect uses consumers, these are not the specific topics that manage the Connect cluster's internal state.
• C. Incorrect. Kafka Connect does not use internal topics for metrics, logs, or traces. Metrics are typically exposed via JMX, and logs are handled by standard logging frameworks like Log4j.
• D. Incorrect. While Kafka Connect works with schemas (usually via an external Schema Registry) and Single Message Transforms (SMTs), it does not store them in internal Kafka topics with these names.

Correct answers: A, C — Place the connector JARs in a directory listed in the plugin.path worker configuration.; Use the Confluent Hub CLI to install the connector.

• A. Correct. Kafka Connect discovers connector plugins from directories configured in the worker's plugin.path setting. Placing the connector JARs (or a directory containing them) in one of those paths allows the worker to load the connector classes during startup.
• B. Incorrect. The Kafka Connect REST API is used for managing connector instances (creating, pausing, deleting, or checking status), not for uploading binary plugin JAR files. Connector binaries must be present on the worker's filesystem before they can be used via the API.
• C. Correct. The Confluent Hub CLI is the standard tool for installing connectors. It automates the process of downloading the plugin and placing it into a directory referenced by the plugin.path configuration.
• D. Incorrect. Kafka Connect runs as a separate framework from the Kafka brokers. Connectors are modular plugins and should never be compiled into the Kafka broker source code.
• E. Incorrect. ZooKeeper handles coordination and metadata for the Kafka cluster but does not manage or load Kafka Connect plugins. Adding JARs to the ZooKeeper classpath has no effect on Connect worker functionality.

Correct answers: A, B — Source connectors can achieve exactly-once by utilizing Kafka's idempotent and transactional producer capabilities.; Sink connectors can achieve exactly-once if the destination system supports idempotent writes or atomic transactions.

• A. Correct. Since Kafka 3.3 (KIP-618), source connectors can achieve exactly-once semantics by leveraging Kafka's transactional producer. This allows the worker to write source records and their corresponding offsets to Kafka atomically.
• B. Correct. For sink connectors, exactly-once is achieved when the destination system can either handle idempotent writes (where multiple writes of the same data have the same effect as one) or participate in atomic transactions with the Kafka offsets.
• C. Incorrect. Exactly-once semantics are not enabled by default. They require specific configuration at the worker level and support from the connector implementation and the external system.
• D. Incorrect. Enabling exactly-once for source connectors requires specific worker-level configurations, such as setting `exactly.once.source.support` to `enabled` in a distributed cluster.
• E. Incorrect. Exactly-once semantics are supported in distributed mode, which is the standard for production environments. It is not limited to standalone mode.

Correct answer: B — org.apache.kafka.connect.transforms.InsertField

• A. Incorrect. MaskField is a Single Message Transform used to obscure or hide the value of an existing field (e.g., for PII or GDPR compliance) by replacing it with a null or a fixed replacement string. It cannot insert new fields into a record.
• B. Correct. InsertField is specifically designed to add new fields to a record. It can inject metadata (such as the topic name, partition, offset, or timestamp) or a static value into the record's key or value.
• C. Incorrect. ReplaceField is used to filter fields (include or exclude) or rename existing fields within a record. It does not possess the functionality to insert new metadata or static values as new fields.
• D. Incorrect. TimestampRouter is a routing SMT that modifies the destination topic name based on the record's timestamp. It affects where the record is sent but does not modify the internal fields of the record itself.
• E. Incorrect. RegexRouter is a routing SMT that uses regular expressions to rewrite the destination topic name. Like TimestampRouter, it manages routing rather than record content modification.

Correct answer: B — It allows testing stream topologies synchronously without needing a running Kafka broker.

• A. Incorrect. The TopologyTestDriver is designed for local, isolated unit testing and specifically eliminates the requirement for a running Kafka cluster, making tests faster and easier to set up.
• B. Correct. The primary advantage of the TopologyTestDriver is that it allows developers to test their stream processing logic synchronously and in isolation without the overhead of a running Kafka broker. This makes it ideal for fast, deterministic unit testing.
• C. Incorrect. The TopologyTestDriver does not automatically generate mock data based on a schema registry. Test records must be manually provided by the developer using the TestInputTopic class.
• D. Incorrect. TopologyTestDriver is an in-memory test utility for logic validation; it does not involve actual network communication or brokers, so it cannot be used to measure network latency or throughput.

Correct answer: C — Send new input records with advanced timestamps via `TestInputTopic`.

• A. `TopologyTestDriver.advanceWallClockTime()` is used to advance the mock wall-clock time specifically for testing punctuators registered with `PunctuationType.WALL_CLOCK_TIME`. It does not advance 'Stream Time', which is what governs standard DSL windowed operations.
• B. Using `Thread.sleep()` is incorrect because `TopologyTestDriver` is a synchronous, single-threaded test utility. It does not track real-world elapsed time, so pausing the test thread will not affect the internal state or clocks of the driver.
• C. In Kafka Streams, windowed operations are driven by 'Stream Time', which is defined as the maximum timestamp seen across processed records. To advance this clock in a test, you must pipe a record with a newer (higher) timestamp using `TestInputTopic`. This triggers window progression and expiration.
• D. `max.poll.interval.ms` is a consumer-side configuration that manages rebalances in a live cluster. It has no functional role in controlling time or windowing logic within the `TopologyTestDriver` framework.

Correct answer: B — Call MockConsumer.schedulePollTask() with a rebalance exception

• A. While MockConsumer.rebalance() updates the internal partition assignment of the mock consumer, it does not automatically trigger the ConsumerRebalanceListener (onPartitionsRevoked/onPartitionsAssigned). In complex unit tests that require simulating the lifecycle of a rebalance during a poll cycle, simply calling rebalance() is often insufficient for testing listener-dependent logic.
• B. Correct. MockConsumer.schedulePollTask() allows you to schedule a Runnable to be executed during the next call to poll(). This is the standard mechanism to simulate events that occur 'during' consumer execution, such as rebalance-related exceptions (like CommitFailedException) or manually invoking the ConsumerRebalanceListener to test how the application handles the transition state.
• C. Changing the consumer group ID is a static configuration change. It does not trigger the internal state machine transitions required to simulate a rebalance event within the context of the Kafka Consumer API's test mocks.
• D. The wakeup() method is used to safely interrupt a blocking poll() call from another thread, causing it to throw a WakeupException. It is used for testing shutdown sequences, not for simulating partition rebalances.

Correct answer: B — Instantiate the custom partitioner class directly and call its partition() method with mock data.

• A. Starting a local Kafka cluster is an integration test approach and is much heavier than necessary for unit testing. While it validates end-to-end behavior, it does not specifically test the custom partitioner logic in isolation.
• B. A custom partitioner implements the Partitioner interface. To unit test it, you should instantiate the class directly and invoke its partition() method with controlled inputs (topic, key, value, and cluster metadata). This allows for deterministic testing of the logic in isolation without a running broker.
• C. TopologyTestDriver is specifically designed for testing Kafka Streams topologies, not for testing producer-side components like custom partitioners.
• D. MockConsumer is a tool for testing consumer-side logic, such as subscription handling. Partitioning is a producer-side responsibility, making the consumer-side mock irrelevant for this task.

Correct answers: A, C — Call MockConsumer.setException(new CommitFailedException()) before the application calls commitSync().; Verify the exception handling logic in your application's catch block.

• A. To test error handling, you must first simulate the error. Although the standard Kafka MockConsumer does not have a literal setException method (often requiring a mocking framework like Mockito in practice), this option represents the necessary step of stubbing the consumer to throw a CommitFailedException during the commitSync() call.
• B. The addEndOffsets() method is used to set the partition end offsets (high watermark) for the mock consumer and is unrelated to simulating commit failures or exceptions.
• C. A complete test must verify that once the CommitFailedException is triggered, the application's catch block logic (e.g., logging, clean-up, or custom retry logic) performs as expected.
• D. The wakeup() method is used to interrupt a consumer that is blocked in a poll() operation, resulting in a WakeupException, not a CommitFailedException.
• E. Enabling auto-commit removes the manual control of offsets via commitSync(), making it impossible to test the specific catch-block logic associated with manual commit failures.

Correct answers: A, C — Instantiate the serializer and call the serialize() method with a test object, then assert the resulting byte array.; Pass the resulting byte array to the corresponding Deserializer and assert that the reconstructed object matches the original.

• A. A serializer can be tested directly by instantiating it, calling the serialize() method with a known object, and asserting properties of the returned byte array. This is a standard unit testing approach that verifies the logic without requiring a Kafka cluster or complex infrastructure.
• B. TopologyTestDriver is specifically designed for testing Kafka Streams topologies in an isolated environment. It is not used for validating the low-level byte array format produced by a custom serializer.
• C. Performing a 'round-trip' test is a robust way to ensure data integrity. By serializing an object and then immediately deserializing it, you can verify that the reconstructed object matches the original, confirming that no data is lost or corrupted during transformation.
• D. The Schema Registry is used for schema management and ensuring compatibility between producers and consumers at the schema level. It is not a tool for unit testing the Java/logic implementation of a custom serializer class.
• E. MockProducer is useful for testing application code that uses a KafkaProducer without needing a real broker. However, it is designed to verify producer behavior (like sending to the correct topic) rather than capturing and inspecting raw network bytes for serializer unit testing.

Correct answer: C — records-lag-max

• A. Incorrect. records-consumed-rate measures the throughput of records per second. While it indicates processing speed, it does not provide information about the gap or backlog relative to the producer's production rate.
• B. Incorrect. fetch-latency-avg measures the time taken to fetch records from the broker. This is useful for diagnosing network issues or broker responsiveness but does not quantify the lag of the consumer group.
• C. Correct. records-lag-max is the primary metric for detecting consumer lag. It represents the maximum difference between the latest offset available in the partition and the offset last committed by the consumer across all partitions. An increasing value indicates the consumer is falling behind.
• D. Incorrect. bytes-consumed-rate measures throughput in terms of data volume per second. Similar to records-consumed-rate, it monitors performance and capacity but does not indicate whether the consumer is keeping pace with the producer.

Correct answer: A — request-latency-avg

• A. Correct. The 'request-latency-avg' metric measures the average time from when a request is sent by the producer until the response is received from the broker. This includes network round-trip time and the time the broker spends processing the request, making it the most direct producer-side metric for confirming broker responsiveness issues.
• B. Incorrect. 'record-queue-time-avg' measures the time records spend in the producer's internal buffer before being sent. While this metric will increase if a slow broker creates backpressure (filling the in-flight request buffer), it is primarily influenced by producer-side settings like 'linger.ms' and 'batch.size' and is not a direct measure of broker processing time.
• C. Incorrect. 'buffer-exhausted-rate' tracks how often the producer's memory buffer is full. This indicates severe backpressure or that the producer is producing data faster than it can be sent, but it does not specifically pinpoint broker processing time as the cause.
• D. Incorrect. 'connection-close-rate' tracks the frequency of connections being terminated. This is generally used for troubleshooting network stability or timeout issues rather than measuring the processing latency of successful produce requests.

Correct answer: B — By calling Topology#describe() and pasting the output into a visualization tool

• A. Incorrect. The __consumer_offsets topic is an internal Kafka topic used to store consumer group offset commits and metadata. It does not contain information about the internal processing topology or logic of a Kafka Streams application.
• B. Correct. In Kafka Streams, calling Topology#describe() returns a TopologyDescription object which provides a textual representation of the processing graph, including sources, processors, state stores, and sinks. This output can be pasted into various third-party visualization tools to render a graphical view of the topology for debugging.
• C. Incorrect. Setting the metrics.recording.level to "TRACE" increases the verbosity and detail of the JMX metrics recorded by the application. While useful for performance monitoring, it does not provide a structural visualization of the processing graph.
• D. Incorrect. Kafka Streams does not have a specific 'kafka-streams-coordinator log' for topology visualization. Standard application logs help diagnose runtime behavior and rebalancing issues but are not the mechanism for describing the processing graph.

Correct answer: B — MetricsReporter

• A. ProducerInterceptor is used to intercept and potentially modify records before they are sent to the Kafka topic. It operates on the message flow and record metadata rather than the internal metrics subsystem.
• B. MetricsReporter is the standard Kafka client interface designed to receive metric updates and report them to external systems. Implementing this interface allows you to push metrics directly to a custom time-series database, bypassing the need for JMX-based scraping tools.
• C. ConsumerRebalanceListener is a callback interface used to react to partition assignment and revocation events during consumer group rebalances. It has no functionality related to collecting or exporting client metrics.
• D. The Partitioner interface determines which partition a producer record is sent to based on the record key or metadata. It influences record routing and load balancing, not observability or metric export.

Correct answers: A, B — record-error-rate; record-retry-rate

• A. The record-error-rate metric is essential for identifying message delivery failures as it measures the average number of record sends per second that resulted in errors. A rising error rate is a direct indicator of persistent delivery problems.
• B. The record-retry-rate metric tracks the average per-second number of retried record sends. It is a critical metric for identifying transient delivery issues and potential broker-side instability, even when messages are eventually delivered successfully.
• C. The batch-size-avg metric measures the average size of the batches sent. While useful for tuning throughput and efficiency, it does not directly track delivery failures or retry logic.
• D. The compression-rate-avg metric measures the average compression efficiency of record batches. It relates to network and storage efficiency rather than the reliability or delivery status of messages.
• E. The waiting-threads metric is not a standard Kafka producer metric for diagnosing delivery failures. It typically indicates thread contention within the application rather than the state of the producer's internal retry or failure mechanisms.

Correct answers: A, B, C — rebalance-rate-per-hour; join-time-avg; sync-time-avg

• A. Correct. The rebalance-rate-per-hour metric provides a direct count of how often the consumer group is triggering reassignments. A high value is the most direct indicator of a rebalance storm, where repeated reassignments interrupt message processing.
• B. Correct. join-time-avg measures the average time a consumer spends in the JoinGroup phase. During a rebalance storm, this metric helps identify the overhead and instability within the group as members repeatedly attempt to rejoin.
• C. Correct. sync-time-avg measures the time spent in the SyncGroup phase, where state is synchronized and assignments are distributed. High or spiky values in this metric confirm that consumers are spending significant time in the 'stop-the-world' portion of the rebalance process.
• D. Incorrect. fetch-size-avg measures the average size of data per fetch request. This is useful for tuning throughput and memory usage but is unrelated to the coordination and rebalancing of consumer groups.
• E. Incorrect. bytes-consumed-rate is a throughput metric. While throughput will typically drop during a rebalance storm, this metric alone cannot distinguish between a rebalance issue, a broker bottleneck, or a decrease in producer activity.
• F. Incorrect. connection-count tracks active network connections to the brokers. While network drops can trigger rebalances, this metric does not provide insights into the internal state or frequency of consumer group reassignments.