Free ISTQB Certified Tester AI Testing (CT-AI) V2.0 Sample Questions

Correct answer: B — Adaptivity

• A. Symbolic logic refers to AI based on formal logic and explicit rules (e.g., if-then statements). It does not typically involve the continuous updating of internal weights through learning from data stream, and thus is not the cause of the described behavior.
• B. Correct. Adaptivity is the ability of an AI system to change its internal state, weights, or behavior based on new data or experience. Since the bot continuously updates its internal parameters from real-time market data, this adaptation leads to behavior changes that may diverge from the state observed during initial pre-deployment testing.
• C. Explainability refers to how well the decisions of an AI system can be understood or interpreted by humans. While a lack of explainability might make it difficult to diagnose why the trades are erratic, it is not the underlying mechanism that causes the behavior to change over time.
• D. Determinism implies that the same input will always produce the same output. The scenario describes a system whose logic evolves over time, which often leads to different outputs for the same market conditions as the internal weights change, signifying a shift away from static deterministic rules.

Correct answer: A — Non-determinism

• A. Correct. Non-determinism refers to a characteristic where the same input can produce different outputs across different executions. In AI systems, particularly generative models like chatbots, the output may vary even when the prompt is identical due to stochastic (random) processes or parameters like 'temperature' that allow for linguistic variety.
• B. Incorrect. Data poisoning is a security vulnerability where malicious data is injected into the training set to influence the model's behavior or accuracy. The scenario describes varying output phrases during testing, not the intentional corruption of the training process.
• C. Incorrect. Model inversion is an attack where an adversary attempts to reconstruct sensitive training data or infer private information by analyzing the model's outputs. This is unrelated to the variability of conversational responses.
• D. Incorrect. Underfitting occurs when a model is too simple to capture the underlying patterns of the data, leading to poor predictive performance. The chatbot's ability to provide multiple relevant (though differently worded) answers is actually a sign of a complex model, not one that has failed to learn the patterns.

Correct answer: A — Historical bias

• A. Historical bias occurs when the data used for training reflects existing human or social prejudices. In this scenario, the model inherits the past discriminatory patterns where fewer loans were granted to specific zip codes, causing it to perpetuate and amplify these historical disparities in its current decisions.
• B. Overfitting occurs when a model learns the training data and its noise too specifically, resulting in poor performance on new, unseen data. While the model is behaving according to the training data, the core issue here is the biased nature of the data distribution itself, not a failure of generalization.
• C. The Oracle Problem refers to the challenge of determining the correct or expected output for a given input, which makes verifying AI results difficult. This scenario focuses on the fairness and bias of the results rather than the difficulty of verifying if the output is correct relative to a specification.
• D. Autonomy refers to the degree to which an AI system can function without human intervention. The discriminatory behavior of the model is a result of the biased patterns learned from the training data, not a measure of how independently the system operates.

Correct answer: A — Fairness

• A. Correct. Fairness is the quality characteristic that addresses bias and discrimination in AI systems. The scenario describes a situation where the model treats a group of applicants differently based on a postal code (often a proxy for demographic data) despite their relevant financial characteristics being equal, indicating a failure in fairness.
• B. Incorrect. Autonomy refers to the ability of an AI system to perform tasks or make decisions without human intervention or with self-governance. The issue described relates to the outcome of the decisions, not the level of human oversight.
• C. Incorrect. Performance Efficiency relates to how a system uses resources such as time, memory, and computing power to perform its functions. The scenario indicates an issue with decision logic and bias, not resource consumption or speed.
• D. Incorrect. Complexity refers to the structural or mathematical intricacy of the AI system and how difficult it is to understand, design, or maintain. While AI models can be complex, this scenario specifically highlights discriminatory behavior, which is a fairness concern.

Correct answers: A, C — LIME (Local Interpretable Model-agnostic Explanations); SHAP (SHapley Additive exPlanations)

• A. Correct. LIME (Local Interpretable Model-agnostic Explanations) is a model-agnostic technique that explains individual predictions by approximating the black-box model locally with an interpretable surrogate model. It helps users understand why a specific output was produced.
• B. Incorrect. SMOTE (Synthetic Minority Over-sampling Technique) is a data pre-processing technique used to address class imbalance by generating synthetic samples for the minority class. It does not provide insights into model decision-making.
• C. Correct. SHAP (SHapley Additive exPlanations) is a framework based on cooperative game theory (Shapley values) that attributes the contribution of each feature to the model's output. It is a widely recognized method for both local and global explainability.
• D. Incorrect. ReLU (Rectified Linear Unit) is an activation function used in neural networks to introduce non-linearity. While it affects model learning, it is not an explainability technique.
• E. Incorrect. K-Means Clustering is an unsupervised learning algorithm used for data segmentation and grouping. It does not provide a mechanism for explaining the predictions of black-box models.

Correct answer: B — Maximizing Recall at the expense of Precision to ensure Reliability in critical detection.

• A. Incorrect. Maximizing Precision focuses on reducing False Positives. The scenario describes minimizing False Negatives (which increases Recall) at the cost of increasing False Positives (which decreases Precision).
• B. Correct. In medical diagnostics, a near-zero False Negative rate is achieved by maximizing Recall (Sensitivity). This ensures that almost no actual cases of cancer are missed. The trade-off is that the system becomes less 'picky,' leading to more False Positives and thus lower Precision.
• C. Incorrect. Accuracy measures overall correctness across both classes but does not capture the specific trade-off between False Negatives and False Positives. Furthermore, Robustness and Explainability are different quality attributes not directly tied to the confusion matrix trade-off described.
• D. Incorrect. Specificity refers to the ability to correctly identify negative cases (reducing False Positives). Sensitivity is another name for Recall. The scenario describes maximizing Sensitivity at the expense of Specificity, whereas this option suggests the inverse. Fairness is also unrelated to this specific classification threshold adjustment.

Correct answer: B — Mitigation of proxy bias.

• A. Data leakage occurs when information from the target variable or data outside the training set 'leaks' into the model during training, leading to unrealistically high performance metrics. It does not relate to fairness or correlated features used for discrimination.
• B. Proxy bias occurs when a seemingly neutral feature (like 'shopping habits') acts as a stand-in for a sensitive or protected attribute (like 'gender') due to high correlation. This allows the model to indirectly discriminate even if the protected attribute is explicitly excluded, violating fairness acceptance criteria.
• C. Inference latency refers to the time taken by an AI model to process input and produce an output. This is a performance-based metric and is unrelated to the issues of fairness, bias, or discrimination described in the scenario.
• D. Differential privacy is a framework for sharing information about a dataset by describing patterns of groups while withholding information about individuals. While it protects data privacy, it does not address biased decision-making resulting from proxy variables.

Correct answer: A — Highly complex models like deep neural networks often achieve higher accuracy but are harder to explain, whereas simpler models like decision trees are highly explainable but may have lower accuracy.

• A. Correct. This accurately describes the 'accuracy-explainability' trade-off. Complex models like deep neural networks can capture intricate patterns (higher accuracy) but act as 'black boxes' due to their architecture. Simpler models like decision trees or linear regression are more interpretable but may lack the capacity to model complex data features, potentially resulting in lower accuracy.
• B. Incorrect. Explainability is more closely related to the complexity of the model architecture and the interpretability methods used, rather than the volume of training data. There is no guaranteed linear decrease in explainability as a dataset grows.
• C. Incorrect. Accuracy and explainability are often competing objectives. A model can become more accurate while remaining opaque; achieving fewer mistakes (accuracy) does not inherently make the logic behind the predictions more transparent.
• D. Incorrect. Explainability requirements are driven by the application context, risk level, and regulatory requirements (e.g., GDPR), not by an arbitrary accuracy threshold. A model with 99% accuracy still requires explainability if it is used in a high-stakes environment like medical diagnosis or legal sentencing.

Correct answer: B — Defining thresholds for the computational efficiency and carbon footprint of training and deploying the AI model.

• A. Incorrect. Green AI focuses on the environmental impact and sustainability of AI processes, not the specific subject matter or domain (e.g., environmental data) used for training.
• B. Correct. Green AI acceptance criteria emphasize computational efficiency, reduced energy consumption, and a smaller carbon footprint during the entire lifecycle of an AI model, including its training and operational phases.
• C. Incorrect. While some models are inherently more efficient than others, Green AI does not mandate specific algorithms like decision trees; instead, it sets performance and sustainability goals regardless of the model architecture.
• D. Incorrect. Although energy-efficient UI designs like dark mode can contribute slightly to power savings, Green AI's primary focus is on the massive system-level energy consumption and emissions resulting from complex model computation and hardware usage.

Correct answer: B — True Positives / (True Positives + False Positives)

• A. Incorrect. This formula defines Recall (also known as Sensitivity or True Positive Rate). It measures the proportion of actual positives that were correctly identified by the model.
• B. Correct. This formula defines Precision (also known as Positive Predictive Value). It measures the proportion of predicted positive identifications that were actually correct, focusing on the quality and reliability of the model's positive predictions.
• C. Incorrect. This formula defines Specificity (True Negative Rate). It measures the proportion of actual negatives that were correctly identified by the model.
• D. Incorrect. This formula defines Accuracy. Accuracy measures the proportion of all correct predictions (both true positives and true negatives) relative to the total number of cases examined.

Correct answer: C — False Discovery Rate

• A. Incorrect. The False Omission Rate (FOR) is the proportion of false negatives among all negative predictions (FN / (TN + FN)). It evaluates the reliability of a negative prediction rather than a positive one.
• B. Incorrect. The False Negative Rate (FNR), also known as the miss rate, is the proportion of actual positives that were incorrectly classified as negative (FN / (TP + FN)).
• C. Correct. The False Discovery Rate (FDR) is the proportion of false positives among all positive predictions made by the model (FP / (TP + FP)). It indicates how many of the predicted positives are actually incorrect.
• D. Incorrect. The False Positive Rate (FPR) represents the proportion of actual negatives that are incorrectly predicted as positive (FP / (FP + TN)). It is based on the total number of actual negative instances, not the total number of positive predictions.

Correct answer: C — By randomly partitioning the data using decision trees; anomalies are isolated closer to the root of the tree with shorter path lengths.

• A. Incorrect. This describes a distance-based outlier detection method, such as k-nearest neighbors (k-NN). Isolation Forest is a partition-based algorithm and does not rely on measuring Euclidean distances between points.
• B. Incorrect. This is a parametric statistical approach (like Z-score) that assumes a specific underlying distribution. Isolation Forest is non-parametric and does not assume the data follows a Gaussian distribution.
• C. Correct. The Isolation Forest algorithm works by randomly partitioning the data using binary trees. Because anomalies are rare and have distinct values, they are easier to isolate than normal points, resulting in them appearing closer to the root of the tree with significantly shorter average path lengths.
• D. Incorrect. This describes reconstruction-based anomaly detection, commonly used in techniques like Principal Component Analysis (PCA) or Autoencoders. Isolation Forest does not involve dimensionality reduction or calculating reconstruction errors.

Correct answers: A, C — It serves as a centralized repository to ensure consistency of feature definitions between offline training and online inference.; It provides low-latency serving of features for real-time online predictions.

• A. Correct. A Feature Store acts as a centralized repository to define, store, and manage features. This ensures that the same feature logic is reused consistently in both training (offline) and inference (online), which helps eliminate training-serving skew.
• B. Incorrect. Hyperparameter tuning is an optimization process performed during model training using specialized tools or AutoML frameworks; it is not a function of a data-centric component like a Feature Store.
• C. Correct. One of the core technical benefits of a Feature Store is its ability to provide low-latency access to features via an online serving layer (often backed by high-performance databases), which is essential for real-time prediction services.
• D. Incorrect. A Feature Store and a model registry are distinct components. A model registry manages trained model versions and metadata, while a Feature Store manages the data (features) used by those models.
• E. Incorrect. Synthetic data generation for class balancing is typically a data preprocessing or augmentation task performed during the data preparation phase. Feature Stores focus on storage, consistency, and retrieval rather than data synthesis during inference.

Correct answer: B — An attacker injects malicious or carefully crafted data into the training dataset to compromise the model's future behavior.

• A. Incorrect. This describes a model extraction or model stealing attack, where the goal is to infer or reconstruct the model's internal parameters or logic by querying its API, rather than corrupting the data.
• B. Correct. Data poisoning involves inserting malicious, mislabeled, or carefully crafted examples into the training dataset. The goal is to corrupt the learning process to influence the model's behavior or degrade its performance in future predictions.
• C. Incorrect. This describes a standard ransomware attack targeting the data pipeline. While it affects the availability of training data, it is not a 'poisoning' attack, which specifically targets the integrity of the data to manipulate model logic.
• D. Incorrect. This describes an adversarial evasion attack, which occurs during the inference (test) phase by applying subtle perturbations to inputs to cause misclassification. Data poisoning occurs during the training phase.

Correct answer: A — Categorical Cross-Entropy.

• A. Correct. Categorical Cross-Entropy is the standard loss function for multi-class classification problems where classes are mutually exclusive. It measures the difference between the predicted probability distribution (from the Softmax layer) and the true distribution, providing strong gradients for effective learning.
• B. Incorrect. Mean Squared Error (MSE) is primarily used for regression tasks. While it can technically be used for classification, it is significantly less effective than cross-entropy because it often results in slower convergence and is less suited to the probabilistic outputs of a Softmax layer.
• C. Incorrect. Mean Absolute Error (MAE) is another loss function used primarily for regression. It does not align naturally with probabilistic class outputs and is not suitable for standard multi-class classification tasks.
• D. Incorrect. Binary Cross-Entropy is used for binary classification (two classes) or multi-label classification (where an instance can belong to multiple classes simultaneously). For standard multi-class classification with a single Softmax output, Categorical Cross-Entropy is the correct choice.

Correct answer: A — To reduce the memory footprint and inference latency by converting floating-point weights and activations to lower-precision integer formats.

• A. Correct. Neural network quantization is an optimization technique that reduces the memory footprint and inference latency by converting high-precision floating-point weights and activations (e.g., 32-bit float) to lower-precision formats (e.g., 8-bit integer). This is crucial for deploying models on resource-constrained edge devices with limited compute power, memory, and battery life.
• B. Incorrect. Encrypting the model's weights is a security measure to protect intellectual property, not the primary goal of quantization. Quantization focuses on operational efficiency and performance rather than security.
• C. Incorrect. This describes fine-tuning or on-device learning. Quantization does not aim to increase accuracy; in fact, reducing precision can sometimes lead to a minor loss of accuracy, which must be managed during the optimization process.
• D. Incorrect. Changing the model's architecture (e.g., from RNN to Feedforward) is a structural modification or transformation. Quantization preserves the existing neural network architecture while changing the precision of the numerical values within it.

Correct answer: B — Feature Engineering

• A. Incorrect. Dimensionality reduction involves reducing the number of input variables in the training data, typically through techniques like Principal Component Analysis (PCA) or feature selection. The scenario describes creating and adding informative variables rather than reducing the total count.
• B. Correct. Feature Engineering is the process of using domain knowledge to create new features from raw data or to transform existing data into representations that better capture patterns for the model. Deriving 'Age_of_Home' and 'Distance_to_City_Center' are classic examples of creating more informative inputs to improve model performance.
• C. Incorrect. Hyperparameter tuning involves adjusting the internal configuration settings of a machine learning algorithm (such as learning rate, number of trees, or regularization strength). The improvement described resulted from changes to the input data representation, not model settings.
• D. Incorrect. Transfer learning involves using a pre-trained model from one task and adapting it to a new, related problem. This scenario describes feature creation from the original raw dataset, not the use of an external pre-trained model.

Correct answer: B — The model is underfitting, indicating high bias.

• A. Incorrect. Overfitting is characterized by low error on training data but high error on validation data, as the model learns noise or specific details of the training set instead of generalizable patterns.
• B. Correct. High error rates on both training and validation sets, coupled with the fact that more data does not improve performance, are classic symptoms of underfitting (high bias). This suggests the model is too simple to capture the underlying structure of the data.
• C. Incorrect. Data leakage usually results in unusually high performance (low error) on validation or test sets because the model has inadvertently 'seen' information it should not have during training. High error rates on both sets contradict this.
• D. Incorrect. Reaching the optimal bias-variance tradeoff would imply the model has achieved the lowest possible total error by balancing complexity and generalization. Persistent high error on both datasets indicates a failure to find this balance, specifically pointing to high bias.

Correct answer: B — It necessitates a shift from expecting exact, reproducible outputs to evaluating outputs within an acceptable probabilistic range or tolerance.

• A. Incorrect. Non-deterministic AI systems involve components like probabilistic models or stochastic training that cannot generally be forced to produce identical outputs for the same input. Attempting to force deterministic behavior is impractical and fails to address the core challenge of validating AI behavior.
• B. Correct. Non-determinism means the same input may lead to different valid outputs. Therefore, testing must shift from checking for exact equality to determining if results fall within an acceptable range, confidence level, or statistical tolerance. This is a fundamental change from traditional 'pass/fail' criteria based on exact expected values.
• C. Incorrect. Regression testing remains essential for AI-based systems. Changes in data, model versions, or environment can significantly affect behavior. If anything, regression testing becomes more important to ensure that model updates do not introduce performance regressions or safety issues.
• D. Incorrect. While white-box testing can be useful, it is not mandated exclusively. In fact, tracing the exact execution path of a complex neural network is often extremely difficult (the 'black box' problem). Non-determinism is primarily managed through outcome-based evaluation and techniques like metamorphic testing.

Correct answer: B — Shadow Testing

• A. Incorrect. A/B testing involves comparing two versions of a system by splitting user traffic or data between them to measure different outcomes. In this scenario, the AI system's outputs are not being used to drive any real-world decisions, so no comparison of outcomes is occurring.
• B. Correct. Shadow testing (or shadowing) involves running a new system in parallel with the existing production system. The new system processes real production data, but its outputs are logged for evaluation and not used for operational decisions. This allows for validation against real-world data without introducing risk to the live process.
• C. Incorrect. A canary release involves gradually rolling out a new system to a small subset of real users to test stability. In a canary release, the system is used operationally for that subset, whereas in this scenario, the AI system's outputs are not used for any live patient decisions.
• D. Incorrect. Metamorphic testing is a test design technique that checks whether outputs change in expected ways when inputs are transformed based on metamorphic relations. It is a verification method rather than a deployment and monitoring strategy like shadow testing.

Correct answer: B — The proportion of neurons in the network that are activated (exceed a certain threshold) by a given set of test inputs.

• A. Incorrect. This describes traditional code coverage for the software implementation that runs or loads the model. Neuron coverage is a model-specific structural metric that focuses on the internal activations of the neural network rather than the lines of source code in the surrounding application.
• B. Correct. Neuron coverage is a white-box testing metric for neural networks that measures the proportion of neurons whose activation values exceed a defined threshold for a specific set of test inputs. It is used to assess how thoroughly the network's internal logic has been exercised by the test suite.
• C. Incorrect. This refers to dataset partitioning and data management during the model's development phase. It is not a testing coverage metric used to evaluate the adequacy of a test suite against a model's internal structure.
• D. Incorrect. The number of hidden layers is a static property of the neural network's architecture (its topology). Neuron coverage is a dynamic measurement of how many neurons are actually 'fired' or activated during execution.

Correct answer: B — The model generates output that contains information neither supported nor contradicted by the source material.

• A. Incorrect. This describes an 'intrinsic hallucination,' where the generated output directly contradicts or conflicts with the specific source material or context provided in the prompt.
• B. Correct. An 'extrinsic hallucination' occurs when the model introduces information that is not grounded in the source material. This content cannot be verified or refuted based solely on the provided context, essentially fabricating details that go beyond the available evidence.
• C. Incorrect. Failure to generate output due to safety filters is a refusal behavior or a safety constraint mechanism, not a hallucination. Hallucinations involve the generation of incorrect or ungrounded content.
• D. Incorrect. Infinite loops or phrase repetition are types of generation degeneration or decoding issues, not extrinsic hallucinations. Hallucinations specifically refer to factual or contextual ungrounding rather than structural repetition.

Correct answer: B — Counterfactual fairness testing

• A. Demographic parity testing is a statistical measure that checks whether different demographic groups receive positive outcomes at similar rates overall. This scenario involves testing the sensitivity of a single instance by changing one attribute, rather than measuring aggregate selection rates across a population.
• B. Counterfactual fairness testing evaluates whether a model's decision remains the same if a sensitive attribute (such as gender, indicated here by the name) is changed while all other features remain constant. By comparing 'John' and 'Jane' on identical resumes, the tester is specifically checking for counterfactual fairness.
• C. Equalized odds testing compares error rates, such as true positive and false positive rates, across different groups to ensure accuracy is consistent. This requires comparing model predictions against ground truth labels for a dataset, which is not what is described in this individual attribute-swap test.
• D. Predictive rate parity (or calibration) requires that the precision or positive predictive value be similar across demographic groups. This is a statistical measure evaluated over a dataset rather than an individual-level 'what-if' test changing a single attribute.

Correct answer: B — The non-deterministic nature of the ML model and its impact on overall system behavior and performance.

• A. Integrating individual software modules into a complete subsystem is a standard software testing concern. While it occurs in AI-based systems, it is not the primary factor that distinguishes AI system testing from traditional system testing.
• B. Correct. AI-based systems often involve ML models that exhibit non-deterministic or probabilistic behavior. System testing must therefore account for how this variability impacts the entire system's functionality, robustness, and performance, which is a significant departure from testing deterministic traditional logic.
• C. Syntax and logic errors in data preparation scripts are usually addressed during component or integration testing of the data pipeline. While important, they do not represent the primary shift in strategy required for system-level testing of an AI application.
• D. User acceptance of the graphical interface is a concern for usability or acceptance testing in both traditional and AI-based systems. It is not the primary differentiator for AI-specific system testing.

Correct answer: B — Gradually rolling out the new ML model to a small subset of users to monitor its performance and stability before a full deployment.

• A. Incorrect. This describes backtesting or offline evaluation. While useful for verifying model performance before deployment, a canary release specifically involves exposing the model to live, real-time traffic.
• B. Correct. A canary release is an acceptance testing and deployment strategy where a new version of an ML model is rolled out to a small subset of users or a portion of live traffic. This allows for monitoring the model's real-world behavior, stability, and business impact before a full-scale deployment.
• C. Incorrect. This describes adversarial robustness testing, which is a security and reliability check. It does not involve the gradual rollout strategy inherent to a canary release.
• D. Incorrect. This describes explainability (XAI) testing, which focuses on interpreting the model's decision-making process using specific frameworks like SHAP or LIME. It is a form of analysis, not a deployment method.

Correct answer: B — Integration Testing

• A. Component testing focuses on testing individual units or modules of the system in isolation. In this scenario, the testing involves the interaction between two distinct components (the inference engine and the feature store), which moves it beyond the scope of isolated component testing.
• B. Integration testing focuses on the interfaces and interactions between different components or systems. Verifying that the inference engine can successfully query the feature store, meet latency requirements (50ms), and handle data-related issues like missing values confirms that these two systems are working together correctly as an integrated unit.
• C. Model evaluation focuses on measuring the predictive performance and quality of the ML model using metrics like accuracy, precision, recall, or F1-score. This scenario focuses on the technical communication and system-level performance (latency and data handling) rather than the statistical quality of the model's predictions.
• D. User Acceptance Testing (UAT) involves validating the entire system against business requirements and user needs, often in a realistic operational environment. This scenario describes a technical verification of the data flow and interface performance between backend components, which is characteristic of integration testing rather than end-user acceptance.

Correct answer: D — Fidelity, utility, and privacy

• A. Precision, recall, and F1-score are standard metrics used to evaluate the performance of classification models on a test set. While synthetic data might influence these metrics eventually, they are not the primary dimensions used to validate the quality or realism of the synthetic data generation process itself.
• B. Bias, variance, and irreducible noise are components of the error decomposition in machine learning used to analyze model generalization. They are not the standard dimensions for assessing whether synthetic data is fit for purpose or representative of the original data distribution.
• C. Volume, velocity, and variety are the classic 'three Vs' of Big Data. They describe the characteristics of data at a high level but do not provide a framework for validating the statistical realism or safety of synthetic data.
• D. Fidelity, utility, and privacy are the standard dimensions for synthetic data validation. Fidelity measures how well the synthetic data resembles the real data distribution (realism); utility assesses how useful the data is for training the intended model; and privacy ensures that sensitive information from the original dataset is not exposed or identifiable.

Correct answer: D — The transformations must be label-preserving, meaning they do not change the ground truth class of the image.

• A. Incorrect. While converting to grayscale is a preprocessing step that can reduce dimensionality, it is not a requirement for valid data augmentation. In many tasks, color is a vital feature, so this transformation could lead to a loss of information and is not a universal critical property.
• B. Incorrect. Increasing image resolution is not a general requirement for data augmentation and is often unrelated to preserving training validity. Upsampling can introduce artifacts and increase computational complexity without improving the model's ability to generalize.
• C. Incorrect. Data augmentation aims to increase variance, but it must not change the semantic meaning of the image. Altering the semantic meaning would invalidate the original label, creating mislabeled data that harms model training.
• D. Correct. The most critical property of data augmentation for classification is that the transformations are label-preserving. If a transformation (like excessive cropping or distortion) changes the image such that it no longer represents its original class, the ground truth becomes incorrect, compromising the integrity of the training data.

Correct answer: B — A sampling rate mismatch causing discrepancies in the generated spectrogram feature dimensions.

• A. Incorrect. Concept drift refers to changes in the statistical properties of the input data or the relationship between input features and labels over time in the real world. This scenario describes a technical mismatch in the data collection and processing pipeline, rather than a drift in the underlying audio concepts.
• B. Correct. A sampling rate mismatch between training (16kHz) and deployment (44.1kHz) data creates a significant discrepancy in the raw signals. Since most speech recognition models convert audio into spectrograms or other frequency-based features, this mismatch can lead to inconsistent feature dimensions or frequency mappings, causing the model to fail to recognize the input correctly.
• C. Incorrect. Data poisoning involves the intentional, malicious manipulation of training data to corrupt the model's behavior. The scenario clearly indicates a technical incompatibility in the capture hardware/software settings, not a security attack.
• D. Incorrect. Overfitting occurs when a model learns noise in the training data too specifically. While overfitting limits generalization, the evidence provided (the specific change in sampling frequency) points directly to an input data pipeline inconsistency rather than the training duration or epoch count.

Correct answer: B — Adversarial testing; exposing a lack of robustness to input perturbations.

• A. Incorrect. Metamorphic testing involves checking whether the system's output changes in a predictable way when inputs are transformed according to specific metamorphic relations. Furthermore, concept drift refers to the statistical properties of the target variable changing over time in production, which is not what is being tested here.
• B. Correct. Adversarial testing involves deliberately introducing small, often imperceptible perturbations to input data to see if the model's output changes significantly. This scenario describes an adversarial attack at inference time, which exposes the model's lack of robustness and its vulnerability to adversarial examples.
• C. Incorrect. Boundary value analysis is a traditional black-box testing technique focused on testing values at the edges of input partitions. While adversarial examples exploit the model's decision boundaries, the specific method of adding noise to fool a classifier is specifically defined as adversarial testing.
• D. Incorrect. Data poisoning involves manipulating the training dataset or the training process itself to introduce vulnerabilities or backdoors into the resulting model. This scenario involves perturbing an input image at inference/test time, not during the training phase.

Correct answer: C — Shadow Testing

• A. A/B Testing involves comparing two versions of a model by splitting real user traffic between them. This is not suitable because both variants' predictions are used to drive decisions, which would risk affecting patient care before the model is proven safe.
• B. Canary Release involves gradually rolling out the new model to a small subset of users. While it limits the scope of potential errors, it still involves using the AI's predictions to influence real outcomes for that subset, which violates the company's safety constraints.
• C. Shadow Testing involves deploying the new model alongside the existing system so that it processes real-time data, but its outputs are not used for clinical decisions. This allows the team to compare the model's performance against ground truth or the existing system in a live environment without any risk to patient safety.
• D. Blue-Green Deployment is a release strategy where you switch traffic between two identical production environments. Once the switch to the 'green' environment occurs, the new model becomes the live system affecting patient care, making it unsuitable for a risk-free evaluation phase.

Correct answer: B — To determine the contribution of each feature to a specific prediction made by the model.

• A. Incorrect. SHAP values are not used for hyperparameter tuning. Hyperparameter optimization is typically performed using methods such as grid search, random search, or Bayesian optimization to improve performance metrics.
• B. Correct. SHAP (SHapley Additive exPlanations) is a game-theoretic approach used to explain the output of machine learning models. It determines the contribution of each feature to a specific prediction, helping testers and developers understand the internal logic of complex ensemble models.
• C. Incorrect. SHAP does not contribute to model compression or deployment optimization. Reducing model size for edge devices is handled by techniques such as pruning, quantization, or knowledge distillation.
• D. Incorrect. SHAP is an explainability tool, not a data generation technique. Generating synthetic test data for minority classes is usually achieved through methods like SMOTE (Synthetic Minority Over-sampling Technique) or GANs.

Correct answer: A — The unintentional use of information from outside the training dataset to create the model, leading to overly optimistic performance estimates.

• A. Data leakage occurs when information that would not be available at prediction time, or information from the test/validation set, is inadvertently included during model training. This allows the model to 'cheat' by finding patterns based on this leaked data, resulting in performance metrics that are overly optimistic and do not generalize to real-world data.
• B. This describes a memory management or resource exhaustion issue (a memory leak) within the software environment. While it is a technical failure, it is unrelated to the machine learning concept of data leakage, which focuses on the integrity of the training and evaluation datasets.
• C. This refers to a privacy or security concern often called 'privacy leakage' or 'PII exposure'. While critical in AI ethics and security testing, the term 'Data Leakage' in the context of ML development specifically refers to the contamination of the training process with information that biases the model's performance evaluation.
• D. The loss of feature columns represents a data quality or ETL pipeline integrity issue. While it affects the model's ability to learn, it is the opposite of data leakage, which involves the inappropriate inclusion of additional, invalid information.

Correct answer: A — A data loading or preprocessing bottleneck occurring on the CPU.

• A. Correct. Low GPU utilization combined with high CPU usage typically signifies that the GPU is idle while waiting for the CPU to provide data. This indicates a bottleneck in data loading, feature extraction, or preprocessing within the training pipeline, where the CPU cannot keep up with the GPU's processing speed.
• B. Incorrect. The learning rate is a hyperparameter that controls how model weights are updated during optimization. While it affects model convergence and training time, it does not directly determine hardware resource utilization metrics like CPU or GPU load.
• C. Incorrect. If the model architecture were too complex for the hardware, it would lead to high GPU utilization as the GPU struggles with the compute load, or it would trigger Out-of-Memory (OOM) errors. Low utilization indicates the GPU is being underfed, not overtaxed.
• D. Incorrect. An excessively large batch size usually increases GPU memory usage and may cause OOM failures. While memory swapping could degrade performance, it does not characteristically present as low GPU utilization paired with high CPU utilization, which is the hallmark of an input pipeline bottleneck.

Correct answer: A — The routing mechanism lacks deterministic stickiness based on a persistent user ID.

• A. Correct. A robust A/B testing framework requires 'stickiness'—ensuring a user is consistently assigned to the same variant for the duration of the experiment. If the assignment is based solely on volatile browser cookies, clearing them resets the assignment. Using a persistent, deterministic identifier (such as a hashed user ID) ensures users remain assigned to the same model variant regardless of session state.
• B. Incorrect. The duration of an A/B test affects the statistical power and the likelihood of detecting an effect, but it has no impact on the technical routing logic or how users are assigned to variants upon returning to the site.
• C. Incorrect. Data leakage typically refers to information from the target variable or test set 'leaking' into the training set, or training information influencing evaluation. While inconsistent routing causes 'experimental pollution,' the root cause described is a routing logic defect, not a training data leakage issue.
• D. Incorrect. Shadow deployment involves processing live requests through a new model variant in parallel with the production model without returning its results to the user. It does not control user-facing traffic routing or assignment between variants.