Subdomain 1.3: Understand the Limitations of GenAI/LLMs
1.A healthcare startup uses a generative AI model to draft patient summaries from clinical notes. The model occasionally inserts incorrect medications and lab values that were not in the original notes. What type of risk does this scenario primarily represent?
- A.The risk of adversarial attacks manipulating model outputs.
- B.The risk of hallucination causing factual errors in critical contexts.
- C.The risk of model inversion leading to data privacy breaches.
- D.The risk of bias from imbalanced clinical training data.
Show answer & explanation
Correct answer: B — The risk of hallucination causing factual errors in critical contexts.
- A. Incorrect. Adversarial attacks involve deliberate manipulation of inputs to deceive the model, which is not described. The issue here is unintentional factual errors, not malicious interference.
- B. Correct. Hallucination refers to the model generating plausible but factually incorrect or fabricated information. In this case, the model inserts non-existent medications and lab values, a classic example of hallucination, which is especially dangerous in healthcare.
- C. Incorrect. Model inversion is a privacy attack where an attacker attempts to recover sensitive training data. The scenario describes factual errors in generated summaries, not leakage of private information.
- D. Incorrect. Bias from imbalanced training data manifests as systematic unfairness or skewed outputs across groups, not random factual errors like incorrect medications or lab values. The core issue here is factual inaccuracy, not bias.