Subdomain 1.4: Implement responsible AI across generative AI and agentic systems
1.Users can upload PDF documents to your generative AI chat application. You need to detect if a user has embedded malicious instructions within the text of the uploaded PDF to manipulate the AI's behavior. Which feature should you implement?
- A.Indirect Prompt Injection detection
- B.Groundedness evaluator
- C.Provenance metadata
- D.Tool-access controls
Show answer & explanation
Correct answer: A — Indirect Prompt Injection detection
- A. Indirect prompt injection detection is specifically designed to identify and mitigate attempts to manipulate an AI's behavior through malicious instructions hidden in external content like uploaded documents, webpages, or emails. This feature is the primary defense against the scenario where a user embeds hidden prompts within a PDF.
- B. Groundedness evaluators assess how well the AI's responses are supported by the provided source data (answer faithfulness). While they ensure the AI isn't hallucinating facts, they do not scan for or detect malicious security instructions embedded in the source documents.
- C. Provenance metadata tracks the origin, history, and ownership of data to support traceability and auditing. It provides context about where a file came from but does not analyze the content of the file for hidden malicious instructions.
- D. Tool-access controls manage permissions and restrict which tools or APIs an AI agent can execute. While this can limit the damage an injection might cause by restricting capabilities, it does not detect the presence of malicious instructions within the uploaded text itself.