Free Microsoft Certified: Operationalizing Machine Learning and Generative AI Solutions (AI-300) Sample Questions

Correct answer: A — Azure Key Vault

• A. Correct. When an Azure Machine Learning workspace is created, an Azure Key Vault is automatically provisioned and associated with it. Azure ML uses this Key Vault to securely store secrets, including the credentials (such as service principal secrets, account keys, or SAS tokens) required to connect to datastores like Azure SQL Database.
• B. Incorrect. While an Azure Storage Account is a required resource for the workspace and can serve as a datastore itself, it is not used to store credentials or secrets. It is primarily used for storing data assets, model artifacts, and environment definitions.
• C. Incorrect. Azure Active Directory (now Microsoft Entra ID) is used for identity and access management (IAM) and authentication to the workspace. However, it is not a storage service for credentials such as database connection strings or secret keys; those are stored in the Key Vault.

Correct answers: A, B, C — Models; Environments; Components

• A. Correct. Models are first-class assets in Azure ML registries. Centralizing model artifacts allows multiple regional workspaces to discover, evaluate, and deploy the same versioned model without manual transfers.
• B. Correct. Environments, which define the software dependencies (Docker images, Conda/pip specs) for a run, are supported registry assets. This ensures that the exact same runtime environment can be reproduced across different workspaces for consistency in training and inference.
• C. Correct. Components represent reusable building blocks or pipeline steps. Registering components centrally enables teams across different regions to compose pipelines using standardized, pre-approved logic and code.
• D. Incorrect. Compute clusters are workspace-scoped infrastructure resources representing physical or virtual compute capacity. They are managed at the workspace level and cannot be stored or shared via a registry.
• E. Incorrect. Datastores are workspace-specific configurations that define how to connect to Azure storage services. While data assets (pointers to files/folders) can be shared in some contexts, datastores themselves are not registry assets.

Correct answer: A — Use `az deployment group create` with a `.bicep` file.

• A. Correct. Using `az deployment group create` with a `.bicep` file is the standard declarative approach for Infrastructure as Code (IaC) in Azure. This method is idempotent, meaning it ensures the environment matches the template state regardless of the starting point, and it integrates natively into CI/CD pipelines such as Azure Pipelines and GitHub Actions for repeatable deployments of the workspace and all its dependencies.
• B. Incorrect. While `az ml workspace create` with a YAML file (Azure ML CLI v2) can be used to configure workspace artifacts, it is not the primary declarative IaC tool for provisioning the full underlying Azure infrastructure (Storage, Key Vault, ACR) in the same way Bicep or ARM does. Bicep is specifically mentioned in the requirement.
• C. Incorrect. The PowerShell cmdlet `New-AzResourceGroup` is used to create a resource group itself, not to deploy a template. Deploying a template via PowerShell requires `New-AzResourceGroupDeployment`. Furthermore, the question specifies the use of Bicep, whereas this option mentions ARM templates.
• D. Incorrect. The `az ml compute create` command is specifically used to provision compute targets (like compute clusters or instances) within an existing Azure ML workspace. It cannot be used to deploy the workspace or its required dependent resources.

Correct answer: A — Use a tool like `git filter-repo` or BFG Repo-Cleaner to rewrite the commit history.

• A. Correct. Tools like `git filter-repo` and BFG Repo-Cleaner are purpose-built to rewrite Git history and purge large or sensitive files from all commits. This is the only way to effectively reduce the repository size after a push by removing the object from the database entirely. Note that after rewriting history, a force push is required, and collaborators will need to re-clone or reset their local branches.
• B. Incorrect. `git commit --amend` only modifies the most recent commit. It does not remove the file if it was part of an earlier commit, nor does it purge the data from the Git history across the entire branch. It would still require a force push and would not solve the bloating for existing history.
• C. Incorrect. The `.gitignore` file only prevents untracked files from being added to the repository in the future. It has no effect on files that have already been committed and are part of the repository's history.
• D. Incorrect. `git revert` creates a new commit that records the removal of the file, but the original commit (containing the 500 MB file) remains in the Git history. Consequently, the repository size remains bloated because Git preserves all historical objects.

Correct answer: A — True

• A. The statement is true because Bicep allows resource IDs to be passed as parameters or retrieved using the 'existing' keyword, which can then be assigned to the storageAccount and keyVault properties of the Azure Machine Learning workspace resource definition. This enables the reuse of existing infrastructure and supports modular deployment strategies.
• B. The statement is false because Bicep and ARM templates fully support referencing external or pre-existing infrastructure via resource IDs; there is no requirement that dependent resources must be created within the same template as the workspace.

Correct answers: C, D — User Identity (Credential-less); Managed Identity

• A. Incorrect. Account Key authentication relies on storing a storage account access key (a secret) within the workspace or its associated Key Vault. It does not utilize the identity of the compute target.
• B. Incorrect. Service Principal authentication involves storing a client ID and secret or certificate. While it uses Azure AD, these credentials must still be managed or stored within the workspace environment, rather than leveraging the compute target's identity.
• C. Correct. User Identity (Credential-less) authentication allows access using Azure AD tokens. This enables the workspace to leverage the identity of the user or a delegated identity flow, avoiding the need to store secrets in the datastore definition.
• D. Correct. Managed Identity allows the compute target (such as an Azure Machine Learning compute cluster or instance) to use its own system-assigned or user-assigned identity to authenticate with ADLS Gen2. This provides a credential-less configuration where no secrets are stored in the workspace.
• E. Incorrect. Shared Access Signature (SAS) authentication uses a generated token that acts as a credential. This token is stored in the workspace and does not utilize the identity of the compute resource.

Correct answer: A — Azure Kubernetes Service (AKS) cluster

• A. Correct. Azure Kubernetes Service (AKS) is the production-standard compute target for real-time inference in Azure Machine Learning. It supports multi-AZ (Availability Zone) cluster configurations for high availability, can be optimized for sub-millisecond latency, and provides robust autoscaling capabilities (via HPA and the cluster autoscaler) to handle variable request volumes, making it ideal for fraud detection.
• B. Incorrect. Azure ML Compute Cluster is primarily designed for distributed training and batch inference workloads. It does not provide the production-grade container orchestration, multi-AZ high availability, or the fine-grained real-time scaling required for low-latency inference endpoints.
• C. Incorrect. Azure Container Instance (ACI) is intended for lightweight development, testing, or low-scale deployments. It lacks support for multiple availability zones, advanced orchestration, and the high-performance autoscaling needed for demanding production-grade real-time inference.

Correct answer: A — az ml online-deployment get-logs

• A. Correct. The 'az ml online-deployment get-logs' command retrieves the logs for a specific online deployment, including the container's stdout/stderr and initialization messages. This is the primary tool for diagnosing startup crashes, such as missing Python dependencies or library version mismatches.
• B. Incorrect. The 'az ml online-endpoint show' command displays the metadata, URI, and provisioning status of an online endpoint. While it can show that an endpoint is in a 'Failed' state, it does not provide the internal container logs needed to troubleshoot the root cause of an initialization failure.
• C. Incorrect. The 'az ml environment list' command enumerates the registered environments in your workspace. While it allows you to verify that an environment exists, it provides no runtime feedback or error output from the deployment container.
• D. Incorrect. The 'az ml online-deployment update' command is used to change the configuration of a deployment that has already been created (such as increasing instance counts). It is not used for viewing startup logs or debugging container crashes.

Correct answer: A — Application Insights

• A. Application Insights is the correct service for this scenario. For Azure Machine Learning real-time endpoints, enabling Application Insights captures request telemetry (including latency) and collects trace logs from stdout/stderr, which allows you to see custom print statements from your scoring script.
• B. Azure Monitor Metrics is focused on collecting and aggregating numeric performance data. While it can track high-level latency metrics, it is unable to capture or display textual logs or custom print statements generated within the application code.
• C. Log Analytics serves as a centralized repository for querying and analyzing log data. While Application Insights data can be stored in a Log Analytics workspace, the specific integration used to instrument and capture application-level traces and request metrics directly from the endpoint is Application Insights.

Correct answer: C — Fairness Assessment

• A. Incorrect. Error Analysis helps identify patterns in model errors and highlights subgroups where the model performs poorly. While it can reveal cohorts with high error rates, it is focused on performance distribution rather than measuring discrimination metrics or assessing bias against protected attributes.
• B. Incorrect. Causal Analysis is used to investigate cause-and-effect relationships between variables. While it can help determine if a sensitive attribute causally influences predictions, it is an exploratory tool for causal inference rather than a systematic assessment tool for quantifying discrimination.
• C. Correct. Fairness Assessment is the component of the Responsible AI dashboard specifically designed to detect and measure discrimination. It uses fairness metrics (such as demographic parity or equalized odds) to evaluate disparities in model outcomes across sensitive groups like gender and age.
• D. Incorrect. Counterfactuals (What-If Analysis) examine how minimal changes to individual input features affect a specific prediction. While useful for explaining individual fairness or local decisions, it is not the primary tool for comprehensive, population-level evaluation of group discrimination.

Correct answer: B — To specify the Python environment and package dependencies required to run the model.

• A. Incorrect. The conda.yaml file does not define input or output data schemas. Data schemas and signatures are typically handled by model metadata within the MLmodel file or separate contract code.
• B. Correct. The conda.yaml file specifies the Python environment, conda channels, and package dependencies required to run the model. This ensures reproducible environments for scoring and deployment, allowing Azure ML to recreate the exact runtime needed for the model.
• C. Incorrect. Serialized model weights and parameters are stored as model artifacts (such as .pkl, .pt, or .onnx files) within the MLflow artifact store, not inside the environment configuration file.
• D. Incorrect. Compute cluster settings and infrastructure configurations for deployment are managed via Azure ML compute targets and deployment configuration objects, not via the conda environment file.

Correct answers: A, B — Archiving a model hides it from default list views in the Azure ML studio.; You can archive a model using the ml_client.models.archive() method.

• A. Correct. Archiving a model in Azure Machine Learning marks it as archived in the registry, which hides it from the default list views in the Azure ML studio. This helps declutter the environment while keeping the model metadata accessible via specific filters or programmatic queries.
• B. Correct. The Azure ML Python SDK (v2) provides the `ml_client.models.archive()` method to programmatically archive specific model names or versions within a workspace or registry.
• C. Incorrect. Archiving only changes the status of the model in the registry; it does not delete the physical artifacts (such as weights or scripts) from the underlying storage account or datastore.
• D. Incorrect. Archiving is a logical organization step and does not affect runtime resources. Existing deployed endpoints referencing the model version will continue to function without interruption.
• E. Incorrect. Archiving is not a destructive action. The version history and metadata remain intact and queryable, and the model can be 'restored' (unarchived) at any time.

Correct answer: B — Application Insights custom dimensions and metrics

• A. Azure Monitor default metrics provide platform-level observability (such as CPU, memory, and network utilization) for the underlying infrastructure. However, they lack the granularity to capture prediction-level telemetry, specific feature distributions, or custom business logic metrics required for deep model monitoring.
• B. Application Insights supports custom dimensions and metrics, allowing you to instrument your model's scoring script to capture prediction inputs, outputs, latencies, and error rates. It provides queryable telemetry via Kusto (KQL), dashboards, and alerting capabilities specifically suited for monitoring real-time model behavior and performance degradation.
• C. The ModelDataCollector is an SDK-level tool used to capture input and output data from a model specifically for data drift analysis and offline auditing. While it collects data for later processing, it is not the primary solution for real-time tracking of custom telemetry metrics and dimensions in production endpoints.
• D. Log Analytics default performance counters focus on host and service-level performance metrics. Although you can ingest custom logs into a Log Analytics workspace, the default performance counters alone are insufficient for tracking model-specific telemetry such as input feature distributions or prediction quality.

Correct answer: B — Feature importance

• A. Incorrect. Wasserstein distance (also known as Earth Mover's Distance) is a metric used to compare probability distributions. While it is frequently used to detect data or distribution drift between training and production datasets, it does not provide insights into the relative importance of specific features or explain the model's decision-making process.
• B. Correct. Feature importance (often calculated via methods like SHAP or LIME) quantifies how much each input feature contributes to the model's predictions. In production monitoring, tracking changes in feature importance helps identify if the model's reliance on specific variables has shifted, which can indicate model decay or underlying data changes that necessitate retraining.
• C. Incorrect. Energy distance is a statistical measure used to compare the distributions of two random vectors. Like Wasserstein distance, it is useful for identifying differences between datasets (drift detection) but does not attribute predictive weight to individual features or explain model logic.
• D. Incorrect. The Pearson correlation coefficient measures the linear correlation between two variables. While it can identify relationships between features and targets, it is limited to linear associations and does not capture the complex, potentially non-linear ways features are utilized by a machine learning model to generate predictions.

Correct answer: A — True

• A. The statement is true because regular monitoring and maintenance are critical aspects of the machine learning lifecycle, allowing for the proactive detection of data drift and anomalies that would otherwise lead to degraded prediction quality.
• B. The statement is false because neglecting model monitoring in a production environment prevents the identification of performance decay, which often occurs as real-world data distributions evolve and differ from the original training data.

Correct answer: B — MedianStoppingPolicy

• A. BanditPolicy is an early-termination strategy based on a slack factor or slack amount relative to the best-performing run. It does not use the median of all runs to determine termination.
• B. MedianStoppingPolicy is specifically designed to stop runs that perform worse than the median of the primary metrics reported by all runs at the same evaluation interval. This built-in policy helps conserve compute resources by abandoning trials that are not performing at least as well as the average (median) of previous runs.
• C. TruncationSelectionPolicy cancels a specific percentage of the lowest performing runs at each evaluation interval based on a truncation percentage. While it targets low performers, it is distinct from the MedianStoppingPolicy which specifically uses the median value as the threshold.
• D. NoTerminationPolicy disables early termination entirely, allowing all hyperparameter tuning runs to complete regardless of their relative performance.

Correct answers: B, C — Specify a distribution object (e.g., PyTorch, TensorFlow, or MPI).; Set instance_count to a value greater than 1.

• A. Incorrect. Serverless endpoints in Azure Machine Learning are used for model inference and serving, not for orchestrating training jobs. Training requires compute targets like compute clusters or serverless training compute.
• B. Correct. Specifying a distribution object (such as PyTorch, TensorFlow, or MPI) is essential for distributed training. It instructs Azure ML on how to orchestrate and synchronize processes across multiple nodes or GPUs using the specific framework's communication backend.
• C. Correct. To scale a training job across multiple compute nodes (horizontal scaling), the instance_count property must be set to a value greater than 1. This ensures that the infrastructure provisions multiple VMs for the task.
• D. Incorrect. The allow_reuse property is a configuration used in machine learning pipelines to determine if a step can skip execution by reusing the output of a previous run with the same inputs; it does not enable distributed training.
• E. Incorrect. Curated inference environments are pre-built Docker images optimized for model deployment and serving. Training jobs require training environments that include specific SDKs, frameworks, and build-time dependencies not found in inference-only environments.

Correct answer: A — True

• A. The statement is true because Azure Machine Learning pipelines provide a mechanism to stitch together different phases of the machine learning lifecycle, such as data preparation and training, into a modular, automated, and reproducible workflow.
• B. The statement is false because the primary purpose of Azure Machine Learning pipelines is to provide orchestration, modularity, and automation for machine learning tasks, rather than just acting as a static script execution environment.

Correct answer: B — Map the flow's input variables to the corresponding columns in the dataset during the run setup.

• A. Renaming columns in the source CSV is an inefficient and brittle approach. It requires modifying the raw data to match a specific flow's internal variable names, which reduces the reusability of the flow and increases the maintenance burden when data sources change.
• B. This is the correct and recommended practice. During the batch run configuration in Azure Machine Learning Prompt Flow, you can explicitly map flow inputs to specific columns in your dataset. This provides flexibility, keeps the prompt flow decoupled from specific data structures, and avoids unnecessary modifications to the source files.
• C. Hardcoding data paths into a prompt template is poor design. It violates the principle of separation of concerns by coupling business logic (the prompt) with infrastructure details (data location), making the flow difficult to port across environments or reuse with different datasets.
• D. Converting the file format is unnecessary because Azure Machine Learning batch runs natively support CSV files. Converting to JSONL would add redundant overhead and time without solving the underlying need for field mapping.

Correct answers: A, C — Working on separate feature branches.; Pulling the latest changes from the remote repository frequently.

• A. Working on separate feature branches isolates changes to prompts or prompt-related code, enabling safe experimentation and structured peer reviews. This practice supports parallel development and ensures the main branch remains stable and auditable for production-ready prompt versions.
• B. Committing directly to the main branch is a poor practice as it bypasses the review process and increases the risk of introducing breaking changes to shared flows. It undermines the stability and governance required in a GenAIOps infrastructure.
• C. Frequently pulling the latest changes from the remote repository ensures your local environment is synchronized with the team's progress. This practice minimizes complex merge conflicts and ensures development is based on the most recent prompt definitions and metadata.
• D. The flow.dag.yaml file defines the logic and structure of the prompt flow; excluding it via .gitignore prevents it from being tracked. This undermines reproducibility and collaboration. Only environment-specific secrets or sensitive data should be excluded from version control.
• E. A shared local working directory leads to concurrency issues, file overwrites, and a lack of traceability. Proper source control relies on individual local clones and pull requests to maintain data integrity and an audit trail of changes.

Correct answer: B — test

• A. Incorrect. The training dataset is used to fit model parameters or optimize prompt structures during the development phase. Because the model or prompt is exposed to this data during construction, using it for evaluation would result in biased, over-optimistic performance metrics and would not accurately represent how the prompt performs on unseen data.
• B. Correct. The test dataset is a held-out, immutable collection of data used to provide an unbiased, final evaluation of a prompt or model version. In GenAIOps and source control workflows, maintaining a consistent and fixed test set allows for reliable benchmarking and regression testing across different prompt versions before they are released to production.
• C. Incorrect. A validation dataset is used iteratively during development to tune hyperparameters or select between multiple prompt candidates. While it helps guide the development process, it is part of the training loop and is not the final held-out benchmark used for the authoritative evaluation of a completed version.

Correct answers: A, B — Serverless API (Pay-as-you-go); Managed Compute (Managed Online Endpoints)

• A. Serverless API (Pay-as-you-go) is a natively available deployment option in the Azure AI Studio Model Catalog (Models as a Service - MaaS). It allows you to deploy models without managing the underlying infrastructure, and billing is based on the number of tokens processed.
• B. Managed Compute (Managed Online Endpoints) is a native Studio deployment option that provisions dedicated virtual machine resources for stable, production-grade inference. It provides control over scaling and instance types, making it suitable for high-throughput or sustained workloads.
• C. Provisioned Throughput Units (PTU) is a reservation model specifically for Azure OpenAI Service proprietary models (like GPT-4) and is not a native deployment mode for third-party open-source models in the general Model Catalog.
• D. Azure App Service is a general web hosting platform for applications. While it can host applications that call an AI model, it is not offered as a native, one-click deployment target within the AI Studio Model Catalog.
• E. Azure Functions is a serverless event-driven compute service. It is not a native deployment platform for hosting foundation models directly from the Azure AI Studio catalog; integrating a model with Functions would require custom packaging.

Correct answer: B — False

• A. The statement is false because serverless API endpoints (Models as a Service) utilize a consumption-based pricing model, typically charging based on the number of tokens processed or requests made, rather than the uptime of dedicated virtual machine instances.
• B. The statement is false because serverless architectures are designed to abstract infrastructure, meaning users do not pay for compute resources when the model is not in use; persistent hourly billing for virtual machine uptime is characteristic of provisioned Managed Online Endpoints, not serverless endpoints.

Correct answer: B — False

• A. The statement is false because while the 'Auto-update to default' policy ensures that your deployment stays up-to-date with the latest service-designated default model version, it does not provide any protection against breaking changes. In fact, model updates often introduce changes in behavior, response formatting, or feature availability that can negatively impact an application's stability.
• B. The statement is false because the 'Auto-update to default' policy inherently risks behavioral or compatibility changes when Microsoft rolls the default version forward. To avoid breaking changes and maintain deterministic behavior in production, Azure recommends pinning the deployment to a specific model version and conducting thorough validation and testing before manually updating to a newer version.

Correct answer: B — az ai hub create-project --name hr-genai-proj

• A. Incorrect. While Azure AI Studio projects are technically specialized Azure Machine Learning workspaces with a 'project' kind, the 'az ml workspace create' command is used for provisioning standalone workspaces and does not support the specific flags --kind project or --hub-id in the context of linked AI Studio project creation.
• B. Correct. The 'az ai hub create-project' command, provided by the Azure AI (ml) extension, is the specific and recommended CLI operation for creating a project within an existing AI hub. This command simplifies the resource-linking process between the hub and the project.
• C. Incorrect. There is no 'az ml project' command group in the Azure CLI. Management of machine learning resources is primarily handled via 'az ml workspace', and AI Studio resources are managed via 'az ai' commands.
• D. Incorrect. The 'az cognitiveservices' command group is used to manage Azure AI Services accounts (such as Speech or Language services). It is not used for managing the lifecycle of Azure AI Studio hubs or projects.

Correct answer: A — kind: 'hub'

• A. Correct. In the Azure Resource Manager (ARM) and Bicep schema for 'Microsoft.MachineLearningServices/workspaces', the 'kind' property serves as the resource discriminator. To define a resource as an Azure AI Studio Hub, you must set 'kind: Hub' (or 'hub'). This allows Azure to differentiate between standard Machine Learning workspaces, AI Studio Hubs, and AI Studio Projects ('kind: Project').
• B. Incorrect. The 'sku' property is used to define the pricing tier and performance characteristics (e.g., 'Basic', 'Premium', or 'Standard') of the resource. It is not used as a discriminator to determine the functional nature of the workspace as a Hub or Project.
• C. Incorrect. 'workspaceType' is not a valid property in the 'Microsoft.MachineLearningServices/workspaces' Bicep or ARM resource schema. While conceptually the resource has a type, the actual field used to specify this in the template is 'kind'.
• D. Incorrect. 'isAiStudio' is not a recognized property in the resource schema. Azure uses the 'kind' metadata field rather than a boolean flag to manage the different personas of the workspace resource.

Correct answers: A, B — Attach the UAMI to the compute instance during creation or update.; Grant the UAMI 'db_datareader' access in the Azure SQL Database.

• A. Correct. To enable the compute instance to authenticate as the UAMI, you must attach (assign) the identity to the resource. This allows the compute instance to request Azure AD tokens for that identity to access other Azure services like Azure SQL and Azure Monitor.
• B. Correct. Authentication is only half the process; the identity must also be authorized. For Azure SQL, you create a contained user for the UAMI and assign it the necessary database-level permissions (such as db_datareader) to allow the script to read evaluation data.
• C. Incorrect. Managed Identities (UAMI or SAMI) do not use client secrets that are visible to or managed by the user. Azure handles the credentials and rotation automatically, which is a key security benefit.
• D. Incorrect. Assigning the 'Owner' role violates the principle of least privilege. The identity only needs specific data-plane access to SQL and monitoring permissions, not full management control over the AI Studio project.
• E. Incorrect. The requirement specifically mandates using a UAMI. While a Service Principal could work, UAMIs are the recommended secure approach for Azure resources as they remove the overhead and risk of secret management.

Correct answers: A, B, C — Hate and fairness; Sexual content; Self-harm

• A. Correct. Hate and fairness is a built-in safety evaluation metric in Azure AI Studio. It helps detect hate speech, biased language, and discriminatory content to ensure the model promotes fairness and does not generate offensive or harassing material.
• B. Correct. Sexual content is a built-in safety evaluation metric designed to identify and flag content that is explicit or inappropriate in nature. This is essential for maintaining professional and safe interactions in sensitive applications like healthcare.
• C. Correct. Self-harm is a built-in safety evaluation metric that identifies content related to self-injury or suicide. In a healthcare context, this is a critical safety measure to ensure the model does not generate harmful suggestions or encourage dangerous behavior.
• D. Incorrect. Groundedness is a quality metric (AI-assisted), not a safety metric. It evaluates how well the model's response is supported by the source information to prevent hallucinations, rather than detecting harmful content.
• E. Incorrect. While Azure AI Studio can evaluate vulnerability to prompt injection, it is categorized as an adversarial or vulnerability metric rather than a content safety metric for detecting offensive output generated by the model.
• F. Incorrect. Coherence is a quality metric that measures how logically consistent and well-structured the model's responses are. It does not evaluate the presence of harmful or offensive content.

Correct answer: A — True

• A. The statement is true because the Groundedness metric specifically measures how well the assertions in the model's generated response are supported by the provided context or source data. It focuses on identifying hallucinations by checking factual alignment with the context, whereas the Relevance metric is used to determine if the response actually addresses the user's question.
• B. The statement is false because Groundedness does not evaluate the intent alignment or task completion relative to the user's query; it is strictly concerned with whether the information in the response can be verified using the provided context.

Correct answer: A — Enable prompt and completion logging in Azure OpenAI diagnostic settings to send data to a Log Analytics workspace.

• A. Correct. Diagnostic settings for Azure OpenAI resources allow you to capture and log request and response payloads (RequestResponse logs). By sending this data to a Log Analytics workspace, you can inspect the exact prompts and completions to debug hallucinations and refine prompt engineering. Note that this should be used carefully with respect to PII and data privacy policies.
• B. Incorrect. Monitoring the 'Generate' metric or similar usage metrics in the Azure AI Foundry dashboard provides quantitative data such as call volume or latency, but it does not capture the text content of the prompts or responses.
• C. Incorrect. The TokenCount metric is a quantitative measure used for tracking throughput, quotas, and costs. It does not provide the qualitative content required to analyze why a model is hallucinating.
• D. Incorrect. Azure Activity Logs are used for auditing control-plane operations (e.g., creating or deleting resources). They do not record data-plane traffic, such as the API payloads containing prompts and model outputs.

Correct answer: A — Prompt Tokens

• A. The Prompt Tokens metric tracks the number of input tokens sent to the model. Since conversation history is included as part of the prompt, excessively long histories will directly increase this metric. Because Azure AI services bill based on the volume of tokens consumed, monitoring prompt tokens allows you to quantify and alert on the cost impact of sending large amounts of input data.
• B. Completion Tokens measure the number of tokens generated by the model as output. While completion tokens contribute to the overall cost, they reflect the model's response rather than the length of the conversation history provided as input.
• C. Provisioned-managed utilization relates to the capacity and throughput utilization of provisioned resources (PTUs) over a period of time. This metric does not provide granular insights into the specific token-level billing impact caused by input payload lengths such as conversation histories.

Correct answer: A — Groundedness

• A. Correct. Groundedness (also known as faithfulness) measures how well the model's generated response is supported by the retrieved source documents. If the LLM includes facts not present in the documents (hallucinations), monitoring and optimizing groundedness through better prompts or retrieval constraints is the direct solution.
• B. Incorrect. Relevance measures how well the response addresses the user's query or how pertinent the retrieved documents are to the query. While low relevance can lead to poor answers, it does not specifically track whether the model is making up facts outside of the provided context.
• C. Incorrect. Fluency evaluates the linguistic quality, grammar, and coherence of the output. A response can be perfectly fluent while still containing hallucinated facts that are not supported by the retrieved context.
• D. Incorrect. Similarity typically refers to the vector closeness between the query and the documents (retrieval quality) or between the model's response and a ground-truth answer. It does not directly quantify the model's adherence to the retrieved source material.

Correct answers: A, B — Reduce the vector dimensions using the `dimensions` parameter in `text-embedding-3` models.; Switch from Exhaustive KNN to HNSW for vector search.

• A. The text-embedding-3 models (small and large) support Matryoshka Representation Learning, which allows for vector dimension reduction via the 'dimensions' parameter. Smaller vectors reduce the computational overhead of similarity calculations and the memory footprint of the index, directly improving retrieval latency in Azure AI Search.
• B. Hierarchical Navigable Small World (HNSW) is an approximate nearest neighbor (ANN) search algorithm that is significantly faster than Exhaustive KNN (brute force) for large datasets. Switching to HNSW reduces the search complexity from O(N) to O(log N), which is critical for solving high-latency bottlenecks in vector retrieval.
• C. Increasing the top-K parameter increases the number of candidates the search engine must retrieve and process. This would increase computational load and network transfer time, worsening the latency bottleneck.
• D. Semantic Ranking is a secondary re-ranking step that uses deep learning models to improve relevance. It adds significant processing time (often several hundred milliseconds) to the retrieval pipeline and is not a technique for reducing latency.
• E. While increasing chunk size reduces the total number of chunks (vectors) in the index, 4096 tokens is extremely large for most RAG applications and can lead to loss of retrieval granularity and accuracy. Reducing dimensionality (Option A) and optimizing the search algorithm (Option B) are more direct and standard methods for addressing search engine latency.

Correct answer: B — Use few-shot prompting to generate JSONL examples and manually review a diverse sample for accuracy.

• A. Training a GAN is not a standard or efficient way to generate labeled text data for entity extraction. GANs typically require large datasets to produce high-quality output and are prone to memorizing sensitive training data, which conflicts with strict privacy constraints.
• B. Few-shot prompting allows a high-capacity model to generate structured training examples (like JSONL) by providing a few high-quality templates. Manually reviewing a sample of this synthetic data is the industry-standard 'human-in-the-loop' step to ensure accuracy and prevent the propagation of hallucinations into the fine-tuned model.
• C. Supervised fine-tuning for extraction requires specific labels (annotations). Feeding raw, unreviewed text without labels would not teach the model which entities to extract. Additionally, using unreviewed medical data risks training the model on sensitive or incorrect information.
• D. QLoRA is a parameter-efficient fine-tuning technique, not a data strategy. Using an open-source generic dataset will not provide the domain-specific medical entity extraction patterns required for this specific task.

Correct answer: B — Merge the new data with the old data and train a completely new model from the original base model.

• A. While Azure OpenAI supports 'continuous fine-tuning' (using a previously fine-tuned model as the base), doing so with only the new data can lead to 'catastrophic forgetting.' The model may optimize for the new edge cases while losing the specific behaviors or accuracy it gained during the first round of training.
• B. This is the recommended approach to ensure model stability and prevent knowledge loss. By merging the original dataset with the new edge cases and retraining from the original base model, the training process optimizes the model weights for the entire distribution of data, maintaining previous knowledge while incorporating new patterns.
• C. LoRA (Low-Rank Adaptation) is an efficient training technique, but Azure OpenAI does not support dynamic LoRA injection into a deployed endpoint at runtime without a standard fine-tuning job. Fine-tuning still requires a structured training process and deployment of a new model version.
• D. While adjusting the system prompt (Few-Shot learning or prompt engineering) can help address specific queries, it does not permanently update the model's weights. It is not a scalable or robust solution for addressing long-term data drift or deep architectural learning of edge cases compared to fine-tuning.