Free ISTQB Certified Tester – Testing with Generative AI (CT-GenAI) Sample Questions

Correct answer: B — Biased data might fail to cover diverse user demographics, leading to untested edge cases.

• A. Incorrect. Bias in synthetic test data does not directly impact the execution time of automated tests. Execution time is primarily influenced by test design, script efficiency, suite size, and the performance of the testing environment.
• B. Correct. Evaluating synthetic test data for bias is crucial because biased data often overrepresents certain groups while underrepresenting others. This lack of representativeness means that tests may fail to cover diverse user demographics and related edge cases, reducing the overall effectiveness and reliability of the testing process.
• C. Incorrect. Token consumption is a factor of the length and complexity of the prompt and the generated output. It is not fundamentally related to whether the content of the data is biased or representative.
• D. Incorrect. While biased data can lead to ethical concerns and potential compliance risks regarding fairness, it does not 'automatically' violate GDPR or CCPA. These regulations focus on the protection, handling, and privacy of personal data, which is distinct from the semantic bias of synthetic data.

Correct answer: A — Enforcing maintainability and code quality standards for AI-generated code.

• A. Correct. AI-generated test code must be subjected to the same review, governance, and quality standards as human-written code. The issues described—brittleness, missing comments, and hardcoded locators—are clear indicators that maintainability standards were not enforced during the generation or integration process.
• B. Incorrect. Temperature settings control the randomness and creativity of the LLM's output. While they affect the content, they do not directly address engineering standards like maintainability or code quality.
• C. Incorrect. While fine-tuning might help the model understand specific framework syntax, it does not inherently solve architectural problems like hardcoded locators or lack of documentation if the overall quality standards are not applied.
• D. Incorrect. Zero-shot prompting is a method used to generate output without providing specific examples. It is a prompting strategy rather than a core principle for long-term code maintenance and quality assurance.

Correct answers: A, B — Pre-training a foundation model requires significantly more computational resources than running inference.; Inference refers to the process of the model generating predictions or text, which incurs ongoing compute costs.

• A. Correct. Pre-training a foundation model involves processing massive datasets (often trillions of tokens) over long periods on large clusters of high-performance GPUs or TPUs. This is the most resource-intensive stage, requiring significantly more total compute than individual inference requests.
• B. Correct. Inference is the operational phase where the model produces outputs based on user prompts. Every request requires the model to perform complex mathematical calculations, which consumes computational resources and results in ongoing operational costs.
• C. Incorrect. Fine-tuning starts from an already trained model and adapts it to a narrower task using a much smaller dataset. Consequently, it requires far fewer computational resources than the initial pre-training phase.
• D. Incorrect. Inference does require computational resources. While a single inference request is less intensive than training the whole model, the hardware must still process the input through the neural network's layers to generate output.
• E. Incorrect. Training costs are generally based on the computational resources (GPU/TPU time) and datasets used during the development phase. It is inference (the usage phase) that is often priced per token processed or generated for the end-user.

Correct answer: A — The model has experienced overfitting during the fine-tuning process.

• A. Correct. This is a classic sign of overfitting, which occurs when a model learns the training data too well—effectively memorizing it—instead of learning generalizable patterns. Because the model has over-optimized for the 50 specific login cases, it fails to adapt or generalize to a new but related task such as password reset.
• B. Incorrect. The temperature setting controls the randomness and diversity of the model's output. While a maximum temperature would lead to more random outputs, it does not explain the perfect reproduction of training data or the structural failure to generalize from a specific dataset.
• C. Incorrect. The context window determines how much input text the model can consider at once. It does not cause a model to overfit or produce nonsensical output specifically for unseen tasks after fine-tuning.
• D. Incorrect. A system prompt provides guidance on style or constraints, but the lack of one would not cause the model to overfit to a small fine-tuning dataset or repeat training examples verbatim at the expense of generalization.

Correct answers: A, C — Intellectual Property (IP) leakage through prompt data.; Vendor lock-in and API rate limiting.

• A. Correct. Intellectual Property (IP) leakage is a primary risk when using public GenAI APIs. Sending proprietary test data, source code, or internal business logic in prompts can expose confidential information to the service provider, potentially violating compliance standards or allowing the data to be used in future training sets.
• B. Incorrect. Public GenAI APIs do not have direct access to execute code on internal corporate databases. Such a scenario would require highly specific, non-standard configurations and explicit permissions that are outside the scope of a typical API integration risk.
• C. Correct. Vendor lock-in and API rate limiting are significant operational risks. Relying on a specific vendor's proprietary API makes migration difficult, while rate limits can throttle execution speed, impacting the scalability and reliability of the automated testing framework.
• D. Incorrect. AI self-awareness is a theoretical concept in science fiction and is not a technical or operational risk associated with current GenAI technologies used in software testing.
• E. Incorrect. A GenAI model does not have the autonomous capability or default permissions to delete external source code repositories. Integration risks generally focus on data privacy and service availability rather than physical destruction of assets.

Correct answer: C — Few-shot prompting

• A. Incorrect. Chain-of-Thought (CoT) prompting involves guiding the LLM to show intermediate reasoning steps or logical sequences to arrive at an answer, rather than simply providing example input-output pairs.
• B. Incorrect. Zero-shot prompting involves asking the LLM to perform a task without providing any prior examples or demonstrations, relying entirely on the model's pre-trained knowledge.
• C. Correct. Few-shot prompting involves providing the LLM with a small number of example input-output pairs within the prompt. This helps the model understand the context, pattern, and desired format of the output.
• D. Incorrect. Meta-prompting refers to the use of higher-level instructions to guide the model on how it should reason or generate prompts itself, rather than using specific input-output pairs for a task.

Correct answer: B — Explore multiple reasoning paths, evaluate choices, and backtrack if necessary to solve complex problems.

• A. Incorrect. The Tree of Thoughts (ToT) framework is a reasoning and problem-solving strategy, not a translation tool for converting test cases into multiple programming languages.
• B. Correct. Tree of Thoughts (ToT) enables the model to explore several reasoning branches or 'thoughts', assess intermediate options, and backtrack when a path is found to be unhelpful. This structured approach helps in solving complex problems that benefit from deliberation and heuristic search rather than a simple linear chain of thought.
• C. Incorrect. ToT is a prompting technique focused on the model's internal reasoning process; it is not a tool for generating graphical visualizations or architectural diagrams of software.
• D. Incorrect. ToT is not designed for token compression. In fact, exploring multiple reasoning branches typically increases computation and token usage compared to simpler prompting methods.

Correct answer: A — Generated Knowledge Prompting

• A. Correct. Generated Knowledge Prompting involves asking the LLM to generate relevant facts or background information about a topic first, then using that intermediate content to ground and improve the accuracy and relevance of the LLM's response to a subsequent task.
• B. Incorrect. Zero-shot prompting involves asking the LLM to perform a task without providing any examples or additional context, focusing on direct task completion rather than a two-step knowledge generation process.
• C. Incorrect. Negative prompting involves specifying what the LLM should not do or include in its response. It is used to exclude certain types of information or behaviors, not to generate supporting knowledge.
• D. Incorrect. Output formatting refers to specifying the structure and presentation of the response (e.g., JSON, tables, or Markdown), not the process of generating intermediate facts for task completion.

Correct answer: B — Chain-of-Thought (CoT) prompting, by instructing the model to 'think step-by-step' through the tax rules before generating tests.

• A. Negative prompting is used to specify what the model should avoid or exclude from its response. While it can help steer the model away from unwanted outputs, it does not provide the logical structure or reasoning necessary to systematically uncover complex edge cases in business logic.
• B. Chain-of-Thought (CoT) prompting encourages the model to decompose a complex problem into intermediate reasoning steps. By instructing the model to 'think step-by-step' through the tax rules, it is more likely to identify the nuances, dependencies, and edge cases required for thorough logical test coverage.
• C. Zero-shot prompting involves asking the model to perform a task without providing specific examples. In this scenario, the tester was already likely using a direct (zero-shot) approach that failed. Removing context would further decrease the model's ability to handle complex domain-specific rules like tax calculations.
• D. Increasing the temperature to 1.0 increases the randomness and diversity of the model's responses. While this might occasionally produce an unexpected scenario, it does not improve the structured logical reasoning needed for comprehensive test coverage and can lead to hallucinations or inaccuracies in a logic-sensitive domain.

Correct answer: A — Providing the model with a small hint or keyword to guide its generation process toward a desired outcome.

• A. Correct. Directional Stimulus Prompting (DSP) involves providing subtle guidance, cues, or keywords to steer the model's output in a specific direction. This technique nudges the model's generation process toward a desired outcome without the need to fully specify the entire response in the initial prompt.
• B. Incorrect. Directional Stimulus Prompting refers to steering the focus or intent of the content, not the physical text direction, script layout, or language format (like right-to-left formatting).
• C. Incorrect. Asking a model to generate its own prompt is a characteristic of self-prompting, prompt refinement, or meta-prompting. DSP relies on external stimuli provided by the user to guide the model's existing logic.
• D. Incorrect. Temperature is a sampling parameter used to control the randomness and creativity of outputs. It is not used with negative values to reverse logic, nor is it the defining feature of directional stimulus prompting.

Correct answer: B — Provide a textual representation of the state transition table or diagram in the prompt, explicitly detailing the '3 failed attempts' rule.

• A. Incorrect. Lowering the temperature makes the output more deterministic and reduces randomness, but it does not provide the missing domain-specific knowledge or business rules that the model is overlooking.
• B. Correct. Providing a textual representation of the state transition table or diagram makes the expected states and transitions explicit. By detailing the '3 failed attempts' rule, the tester provides the necessary context and constraints to ensure the LLM generates accurate test cases for that specific branch of logic.
• C. Incorrect. While personas can influence the style or tone of a response, they do not reliably improve the coverage of complex functional logic or specific business rules. A persona is not a substitute for explicit requirements.
• D. Incorrect. Zero-shot prompting relies on the model to infer rules without specific examples or detailed guidance. Since the model is already failing to identify the transition, continuing with zero-shot prompting is unlikely to resolve the issue; the model requires more structured input.

Correct answer: B — Increase the temperature.

• A. Incorrect. Decreasing the temperature makes the model more deterministic and predictable. This typically leads to less variety and more repetition of high-probability tokens, which would worsen the problem.
• B. Correct. Increasing the temperature introduces more randomness into the model's output by flattening the probability distribution of next tokens. This results in more varied, diverse, and less predictable test data, which is ideal for generating different names and addresses.
• C. Incorrect. The maximum token limit controls only the length of the generated response. It does not affect the diversity or randomness of the content within that output.
• D. Incorrect. While a frequency penalty can reduce the repetition of specific words within a single completion, increasing the temperature is the standard and more effective way to improve the overall variety and creative range of the entities generated across the entire output.

Correct answer: B — Role prompting.

• A. Few-shot prompting involves providing one or more specific examples of input-output pairs to guide the model. This refined prompt does not include any examples of test cases, so it is not using few-shot prompting.
• B. Role prompting (also known as persona prompting) involves assigning a specific role, persona, or expertise level to the GenAI model to narrow the response style and domain focus. In this case, assigning the role of a 'Senior Security QA Engineer with 10 years of experience' improves the depth and specialization of the generated test cases.
• C. Chain-of-thought prompting involves guiding the model through a step-by-step reasoning process or asking it to explain its logic. While the prompt focuses the scope on security, it does not request a step-by-step reasoning or thought process.
• D. Output formatting involves specifying the structure or visual format of the response, such as requesting tables, bullet points, or specific data formats like JSON. This prompt refines the perspective and subject matter focus, not the layout of the response.

Correct answers: A, C — Accountability; Fairness

• A. Accountability is a foundational Responsible AI principle. It ensures that organizations and developers are responsible for the decisions and actions of their AI systems, requiring them to explain, justify, and take responsibility for outcomes to promote transparency and trust.
• B. Maximum Profitability is a business objective focused on financial return. While a standard corporate goal, it is not an ethical or foundational pillar used to guide the responsible and safe deployment of AI systems.
• C. Fairness is a universally recognized pillar of Responsible AI. It focuses on ensuring that AI systems avoid unjust bias and do not discriminate against individuals or groups, thereby promoting equity and justice across AI-driven processes.
• D. Planned Obsolescence is a product strategy involving the intentional shortening of a product's lifespan. It is unrelated to the ethical principles of governance, safety, and human oversight central to Responsible AI.
• E. Vendor Lock-in is a commercial dependency issue where a customer becomes restricted to a single provider. While it represents a business risk, it is not considered an ethical governance pillar within Responsible AI frameworks.

Correct answer: A — ISO/IEC 22989

• A. Correct. ISO/IEC 22989 is specifically designed to provide foundational concepts and terminology for artificial intelligence. It serves as a common reference and vocabulary for stakeholders like testers, developers, and regulators to ensure consistency and a shared understanding across the AI ecosystem.
• B. Incorrect. IEEE 802.11 is the set of standards for wireless local area networks (Wi-Fi) and does not address artificial intelligence concepts or terminology.
• C. Incorrect. ISO/IEC 33001 is part of a series focused on process assessment and capability/maturity models. It is not specifically related to defining artificial intelligence terminology.
• D. Incorrect. RFC 793 is the technical specification for the Transmission Control Protocol (TCP), a core networking protocol, and has no connection to AI terminology or foundational AI concepts.

Correct answer: C — Temperature

• A. Incorrect. Top-p (Nucleus sampling) controls the range of next tokens by selecting the smallest set whose cumulative probability reaches p. While lowering top-p can influence diversity and reduce the pool of candidates, it is not the primary or most common hyperparameter used to enforce determinism.
• B. Incorrect. Frequency penalty is used to reduce the likelihood of repeating the same words or phrases. While it can improve the variety of the text, it does not primarily make the model more deterministic or focused on the most probable tokens.
• C. Correct. Temperature is the standard hyperparameter used to control the randomness of the model's output. By setting the temperature to a lower value (approaching zero), the probability distribution becomes more peaked, making the model pick the most likely next tokens. This makes the output more deterministic and focused, thereby reducing the likelihood of 'creative' hallucinations.
• D. Incorrect. Presence penalty encourages the model to discuss new topics by penalizing tokens that have already appeared. It affects the novelty and diversity of the content but does not make the output more deterministic.

Correct answer: B — Confirmation bias

• A. Algorithmic bias refers to systematic and repeatable errors in a computer system that create unfair outcomes, often affecting specific groups. This scenario does not involve unfair treatment or systematic skew in the model's core logic, but rather an interaction where a leading prompt steers the output.
• B. Confirmation bias occurs when the user's prompt is framed to support an existing hypothesis, and the GenAI responds by reinforcing that belief rather than providing an objective analysis. In this case, the developer's leading question caused the tool to support a mistaken theory, which can mislead the diagnosis of the defect.
• C. Extrinsic hallucination refers to the AI generating information that is not grounded in the input data or verifiable facts. While the AI is technically hallucinating an explanation, the specific human-AI interaction risk described—where the user influences the AI to agree with their theory—is best categorized as confirmation bias.
• D. Representation bias occurs when the training data underrepresents certain concepts, groups, or cases, leading to skewed model behavior. This scenario is about a prompt-driven reinforcement of a specific user belief rather than a lack of representation in the underlying training set.

Correct answer: B — Data poisoning

• A. Prompt injection involves manipulating inputs to an AI model at inference time to produce unintended outputs or bypass safety filters. It does not involve modifying the underlying training or fine-tuning data itself.
• B. Data poisoning involves tampering with the training or fine-tuning dataset to introduce vulnerabilities, biases, or incorrect behaviors into the AI model. Modifying the repository of test cases used for fine-tuning is a primary example of this security risk.
• C. Shadow AI refers to the unauthorized use of AI tools or models within an organization without official approval or oversight. The scenario describes an attack on the integrity of a sanctioned fine-tuning pipeline, not the usage of unauthorized tools.
• D. API key leakage refers to the unauthorized exposure of credentials used to access AI services. The scenario focuses on the malicious modification of the test-case repository rather than the exposure of secret credentials.

Correct answer: B — Enterprise tiers provide guarantees of data isolation and non-training on user prompts

• A. Incorrect. Faster test case generation is a performance or productivity benefit, not a data security advantage. Security-specific concerns focus on the protection, privacy, and unauthorized use of data.
• B. Correct. Enterprise GenAI tiers typically provide technical and contractual assurances that user prompts and organizational data are isolated and, crucially, not used to train the public foundation models. This ensures that sensitive testing data remains confidential and proprietary information is not leaked into the shared model's knowledge base.
• C. Incorrect. The use of API keys is an authentication and access control mechanism common to both public and enterprise tiers. It does not provide the primary data privacy distinction required between the two tiers.
• D. Incorrect. Automated bug fixing is a functional capability and does not relate to data security, privacy, or the methodology of how user input is handled or isolated from global model training.

Correct answer: B — Shadow AI

• A. Incorrect. An insider threat refers to a person within an organization who intentionally or accidentally causes a security risk. While the testers are insiders, this term is too broad for the specific unauthorized usage of AI tools outside of approved governance.
• B. Correct. Shadow AI refers to the unauthorized or unsanctioned use of AI tools by employees within an organization without the knowledge or approval of the IT or security departments. Using free GenAI tools on personal devices to create test scripts against policy is a textbook example of shadow AI.
• C. Incorrect. BYOD refers to the practice of employees using their own devices for work purposes. While the scenario mentions personal devices, the specific phenomenon of using unvetted software/AI tools is 'Shadow AI', not 'BYOD optimization'.
• D. Incorrect. Agile testing is a software testing practice aligned with agile development methodologies. It focuses on continuous testing and fast feedback and is unrelated to security risks or unauthorized tool usage.

Correct answer: B — Craft concise prompts and strictly limit the `max_tokens` output parameter

• A. Incorrect. Using maximum token limits for every request increases the computation required during inference. Since energy consumption is proportional to the volume of tokens processed and generated, this leads to a higher environmental impact.
• B. Correct. Energy consumption during inference is directly related to input and output token counts. Crafting concise prompts reduces input processing work, and strictly limiting the `max_tokens` parameter minimizes the generation phase, leading to lower computational costs and energy usage.
• C. Incorrect. Including redundant context increases the size of the input prompt. Processing unnecessary tokens increases the inference workload and energy consumption without contributing to efficiency.
• D. Incorrect. Generating multiple alternative responses requires significantly more computation than producing a single response. Discarding unused outputs does not recover the energy already expended during the generation process.

Correct answer: B — E-waste generation due to hardware obsolescence

• A. Deforestation is primarily driven by factors such as agricultural expansion, logging, and urbanization. While supply-chain activities may have indirect impacts, it is not the primary environmental issue directly resulting from the rapid turnover and release of GenAI hardware.
• B. The rapid pace of innovation and the frequent release of newer, more powerful GPUs for GenAI training and inference lead to shorter hardware lifecycles. As older equipment becomes obsolete more quickly, it contributes significantly to the global electronic waste (e-waste) problem.
• C. Ocean acidification is mainly caused by the absorption of atmospheric carbon dioxide (CO2) by the oceans. While the energy used to power GenAI contributes to carbon emissions, the specific issue of hardware advancement and obsolescence is linked to e-waste rather than acidification.
• D. Depletion of the ozone layer is caused by the release of specific ozone-depleting substances (ODS), such as chlorofluorocarbons (CFCs), not by the hardware upgrade cycles associated with GenAI development.

Correct answer: B — Implementing edge computing to perform inference locally on the device

• A. Upgrading cloud servers to faster GPUs may improve processing speed at the data center, but it does not address the energy consumed by transmitting data across the network, which is the source of the overhead described.
• B. Edge computing moves the GenAI inference closer to where the data is generated (on the device or local gateway). By processing data locally, the need to transmit data to the cloud is significantly reduced or eliminated, thereby lowering network traffic, latency, and energy consumption associated with communication.
• C. Increasing payload size to send fewer packets may slightly reduce network protocol overhead, but it does not eliminate the need for transmission. In many cases, it increases latency and still consumes substantial network energy compared to local processing.
• D. Cellular connections generally consume more energy than Wi-Fi for data transfers. Switching to cellular would likely increase the energy footprint of the device rather than reduce the network energy overhead.

Correct answer: B — LLMOps introduces new operational requirements such as prompt management and token cost tracking.

• A. Incorrect. LLMOps does not exclusively focus on training models from scratch. In many cases, it operationalizes pre-trained foundation models through prompting, fine-tuning, and deployment, encompassing the entire model lifecycle.
• B. Correct. LLMOps introduces specific operational requirements that go beyond traditional MLOps, such as prompt versioning/management, safety guardrails, and the tracking of token usage and inference costs, which are critical due to the unique economics and behavior of LLMs.
• C. Incorrect. LLMOps does not eliminate the need for continuous monitoring; it emphasizes it. Monitoring is vital to detect quality drift, hallucinations, safety issues, and changing usage patterns in production.
• D. Incorrect. Large Language Models often require substantially more computational infrastructure than traditional machine learning models, especially regarding GPU resources for inference, vector databases for RAG, and extensive logging/observability.

Correct answer: B — Fine-tune the model on a dataset of previous contracts to internalize the specific writing style and tone.

• A. Incorrect. RAG is primarily used to provide external factual knowledge and grounding during generation. While it can retrieve relevant text snippets, it is not the most effective method for internalizing and consistently reproducing a nuanced writing style or corporate tone.
• B. Correct. Fine-tuning is the preferred method for style adaptation. By training the model on a specific dataset of previous contracts, the LLM learns the specific vocabulary, phrasing patterns, and stylistic nuances required to mimic the firm's corporate tone.
• C. Incorrect. While prompt engineering can influence style, it is often less reliable for achieving a highly specific and consistent corporate tone than fine-tuning. Furthermore, fine-tuning is specifically used to alter a model's behavior and tone.
• D. Incorrect. Training a foundation model from scratch is extremely resource-intensive, expensive, and unnecessary for this use case. Fine-tuning an existing model is a much more efficient and practical way to achieve stylistic customization.

Correct answer: B — Catastrophic forgetting.

• A. Incorrect. Data drift refers to changes in the statistical properties of input data over time in a production environment. While it causes model degradation, it does not describe the specific loss of previously learned capabilities immediately following a fine-tuning process.
• B. Correct. Catastrophic forgetting occurs when a model is trained on new information (fine-tuning) and the weight updates essentially overwrite the information required to perform previously learned tasks, such as basic math in this scenario.
• C. Incorrect. Prompt injection is a security vulnerability where malicious input is used to manipulate the model's behavior at inference time. This scenario describes a change in the model's internal knowledge base due to training, not an adversarial attack.
• D. Incorrect. Overfitting occurs when a model learns the specific details and noise in its training data (the medical dataset) to the extent that it negatively impacts performance on new data. The scenario describes the loss of a separate, general skill (math), which is specifically defined as catastrophic forgetting.

Correct answer: A — A Human-in-the-Loop (HITL) approval gate before the agent executes state-altering actions.

• A. Correct. A Human-in-the-Loop (HITL) approval gate is a critical safety mechanism for autonomous agents. It ensures that a human reviewer must authorize state-altering or destructive actions before execution, providing a definitive control layer to prevent unintended damage in sensitive or shared environments.
• B. Incorrect. A larger context window improves the model's ability to process and recall information (like documentation), but it does not serve as a safety barrier or execution control. It does not prevent the agent from deciding to take a destructive action.
• C. Incorrect. Changing the deployment model from cloud-based to local may address privacy or latency requirements, but it does not inherently limit the agent's ability to perform destructive actions on the target application. Control depends on guardrails and approval logic, not the model's location.
• D. Incorrect. Increasing the token generation limit allows the agent to produce longer outputs or more complex scripts, but it does not add any architectural protection or safety oversight regarding the actions those scripts perform.

Correct answer: A — Model Routing, using a smaller, cheaper model for the mock data and a larger, more capable model for the test strategies.

• A. Correct. Model Routing (or model cascading) is a suitable approach here because it allows the platform to match the task complexity to the model's capabilities. Simple, high-volume tasks like generating mock names are sent to smaller, faster, and cheaper models, while complex, low-volume tasks like devising test strategies are sent to more powerful models. This balances cost, latency, and quality.
• B. Incorrect. Using a single massive model for both tasks is inefficient. It results in unnecessary costs and computational overhead for simple tasks where a smaller model would suffice. Fine-tuning a massive model does not solve the fundamental issue of over-provisioning for simple generation tasks.
• C. Incorrect. A vector database is primarily used for retrieval and similarity search (RAG), not for generating new mock data or planning complex test strategies. This approach does not address the core need for LLM-based generation and strategy design.
• D. Incorrect. Human-in-the-Loop (HITL) can improve accuracy for high-stakes outputs but is counterproductive for generating thousands of simple mock user names. It would introduce significant manual effort, increase costs, and slow down the process without providing a justifiable benefit for low-complexity data.

Correct answer: A — Implement a self-reflection loop where the subsequent failure and stack trace are fed back to the LLM for iterative correction up to a maximum retry limit.

• A. Implementing a self-reflection loop is a key architectural pattern for improving the reliability of LLM-powered systems. By feeding the failure information (such as stack traces) from a failed repair attempt back to the LLM as additional context, the model can reason about its previous error and refine its next suggestion. This iterative correction significantly increases the likelihood of a successful fix compared to a single-shot attempt.
• B. Static code analysis tools identify patterns of defects but lack the context-aware, generative capability needed to provide dynamic fixes for broken UI tests. Replacing the LLM with static analysis would limit the system's ability to handle complex UI failures and does not improve the existing repair workflow.
• C. Increasing the LLM temperature introduces more randomness and variance into the output. While this makes the model more 'creative,' it reduces the precision and reliability of code repairs. For fixing UI tests, deterministic and feedback-driven iteration is far more effective than increasing stochastic randomness.
• D. A semantic cache is useful for efficiency and cost reduction by avoiding the repetition of previous mistakes in future runs. However, it does not directly increase the immediate success rate of a repair session. The tool currently stops after one failure; to improve success, it needs a mechanism to try new, informed approaches during the current session (an iterative loop).

Correct answers: B, D — A high level of enthusiasm for exploring and learning new AI technologies.; A willingness to share knowledge and mentor peers on new tools.

• A. Resistance to change is counterproductive for a champion role. GenAI Champions should help promote and embrace new ways of working and innovative methodologies, rather than blocking them.
• B. High enthusiasm is a crucial trait for champions. They need to model curiosity and openness to help build momentum and encourage other team members to adopt GenAI technologies.
• C. The role of a champion is to influence and support others. Working in isolation is detrimental to change management, which requires active collaboration and communication to be successful.
• D. A willingness to share knowledge and mentor peers is essential for a champion. This helps disseminate new skills across the organization, reduces resistance, and ensures the team can effectively use new GenAI tools.
• E. Strict adherence to legacy manual processes suggests a reluctance to adapt. Champions should support transformation and continuous improvement rather than clinging to outdated methods that GenAI aims to enhance or replace.

Correct answer: B — It poses severe security and data privacy risks; it should be addressed by establishing clear guidelines on approved tools and providing secure, sanctioned alternatives.

• A. Incorrect. The primary concern with Shadow AI is not financial savings on licenses, but the lack of governance. Banning all free tools is an overly extreme measure that fails to address the underlying employee needs and risks associated with non-compliant usage.
• B. Correct. Shadow AI creates significant security, confidentiality, and data privacy risks because employees may input sensitive or proprietary information into public or unvetted tools. The appropriate change management response is to establish clear usage policies, approve secure tools, and provide sanctioned alternatives that allow users to remain productive without bypassing organizational controls.
• C. Incorrect. The core risk of Shadow AI is not related to the image of the IT department. Punitively firing employees is an ineffective management response that does not solve the root cause of the issue, which is typically a gap in provided tools or education.
• D. Incorrect. While any tool usage has an impact on infrastructure, network congestion is not a primary risk of GenAI adoption. The central concern is the ungoverned handling of data and lack of oversight regarding output quality and compliance.

Correct answer: B — Roles will evolve to include 'AI Test Engineers' who blend domain expertise, testing fundamentals, and AI tool orchestration.

• A. Incorrect. While prompt engineering becomes a valuable skill within the tester's toolkit, it does not replace core testing principles. Testers are expected to broaden their capabilities and use AI as an enabler rather than abandoning the foundations of software quality.
• B. Correct. The integration of GenAI leads to the emergence of hybrid roles, such as 'AI Test Engineers.' These roles require a synergy of domain knowledge, traditional testing methodology, and the technical ability to orchestrate and validate AI-driven testing processes.
• C. Incorrect. AI does not eliminate the need for testers; it transforms how they work. Human oversight, critical thinking, risk assessment, and complex validation remain essential, meaning the role is redefined rather than removed.
• D. Incorrect. Hardware maintenance is an infrastructure and operations task, not a redefinition of the testing role. Testers will continue to focus on quality-related activities, including the strategy and oversight of AI-assisted testing.

Correct answer: B — A comparison of the average time taken to design test cases before and after implementing GenAI.

• A. Incorrect. The number of models evaluated reflects the effort put into the selection and procurement phase, but it does not measure the actual operational impact or efficiency gains realized within the testing process after adoption.
• B. Correct. This is a direct measure of productivity and efficiency. By comparing the 'before and after' cycle time for core testing activities like test design, organizations can quantitatively determine if GenAI has reduced effort and saved time.
• C. Incorrect. Measuring lines of code is a volume metric. A higher volume of code does not necessarily translate to efficiency; it could represent low-quality output or bloat, and does not account for the time spent reviewing or debugging that code.
• D. Incorrect. The number of prompts indicates the intensity of interaction or usage levels, but it doesn't provide insight into whether those interactions resulted in faster or better testing outcomes.

Correct answer: B — Incorporate a mandatory technical review step to validate AI-generated test steps against official system documentation.

• A. Implementing automated, unverified execution of AI suggestions is highly risky and inefficient. It leads to wasted testing efforts, false defects, and potential system disruptions. Hallucinations must be mitigated through human-in-the-loop validation rather than blind trust.
• B. This is the correct approach for an adoption roadmap. A mandatory technical review step ensures AI-generated test steps are checked against authoritative sources like API specifications, architecture diagrams, and system documentation. This governance step reduces the risk of hallucinations while still allowing the team to benefit from GenAI assistance.
• C. Updating the system to include hallucinated endpoints is impractical and logically backwards. The system architecture should be driven by business requirements, not by the inaccuracies of a generative model. The solution lies in constraining and validating the AI output, not altering the product to match its errors.
• D. Immediately terminating the GenAI tool is a disproportionate response to a common and manageable issue. The purpose of a pilot phase is to identify such limitations and establish the necessary guardrails and verification controls to safely integrate the tool into the workflow.

Correct answer: B — They should be treated as code, version-controlled, and stored in a shared repository.

• A. Incorrect. Keeping prompts as personal knowledge is not scalable, prevents organizational learning, and creates a single point of failure. Mature organizations focus on making effective prompts reusable assets rather than individual secrets.
• B. Correct. In a mature GenAI testing environment, highly effective prompts are treated as strategic assets. Managing them like code—using version control, peer reviews, and shared repositories—ensures traceability, consistency, reuse, and continuous improvement across the organization.
• C. Incorrect. Deleting prompts after use would lead to a significant loss of intellectual property and require repeated effort to recreate effective results. Bias management is handled through governance and validation, not by discarding successful engineering efforts.
• D. Incorrect. Relying exclusively on a third-party vendor for prompt management reduces organizational control over quality, security, and specific testing needs, and it creates unnecessary vendor dependency.