What you will be able to do
- Find where traces and usage records for each Cortex feature are stored
- Explain how LLM-as-a-judge scores an AI application's output
- Use usage views and guardrail signals to monitor security and data quality
- Lock down Cortex Analyst with database roles and semantic-view privileges, and audit its generated SQL
- Govern Cortex Agent tools and orchestration
1.Where AI traces live
AI Observability answers audit questions such as what happened in a specific production request and whether Guardrails blocked it. Different Cortex features keep this evidence in different places. Before you can trace sensitive data through a Gen AI application, you need to know where each feature writes its records.
| Feature | Traces / request logs | Account Usage view |
|---|---|---|
| Cortex Agents | SNOWFLAKE.LOCAL.AI_OBSERVABILITY_EVENTS | CORTEX_AGENT_USAGE_HISTORY |
| Cortex Analyst (direct requests) | SNOWFLAKE.LOCAL.CORTEX_ANALYST_REQUESTS_RAW | CORTEX_ANALYST_USAGE_HISTORY |
| Built-in functions (AI_COMPLETE, AI_CLASSIFY) | None written | CORTEX_AI_FUNCTIONS_USAGE_HISTORY |
| Cortex REST API | None written | CORTEX_REST_API_USAGE_HISTORY |
| Cortex AI Guardrails | Flagged scans appear in Cortex Agent monitoring traces | CORTEX_AI_GUARDRAILS_USAGE_HISTORY |
An agent trace is made up of conversation threads, turns and execution spans. During a run, the agent emits events showing its reasoning, the tool calls it made and its reflections. That makes the trace your audit trail for sensitive data. You can see which tool was called, what came back, and how the agent used it in the answer. When an agent calls Analyst as a tool, those steps are part of the agent trace. Direct Analyst calls are logged elsewhere.
For a custom application, such as a RAG pipeline on Snowpark Container Services or an app hosted outside Snowflake, use TruLens. Snowflake registers the application as an External Agent object that holds metadata only. Its traces and scores go to AI_OBSERVABILITY_EVENTS.
Checkpoint 1 of 10· Exam question
A nightly job runs `INSERT INTO summaries SELECT id, SNOWFLAKE.CORTEX.COMPLETE('mistral-large2', prompt) FROM tickets` over 2 million rows. One unsafe or failing row aborts the whole statement. Security wants unsafe model output blocked and the batch to continue. What should the engineer change?
Correct answer: C — Call TRY_COMPLETE with an options object setting `'guardrails': true`, so filtered or failed rows return NULL and a later predicate can drop them.
- A. Incorrect: setting `guardrails` to false disables Cortex Guard, and a regular expression is not the Cortex Guard model. Such a pattern filter would miss most harmful content.
- B. Incorrect: an EXCEPTION block works per statement, not per row, so one rejected row still aborts the INSERT. It also cannot retry a single row inside a set-based SELECT.
- C. Correct: TRY_COMPLETE returns NULL instead of raising an error, and the `guardrails` option enables Cortex Guard filtering. Filtered rows become NULL, so the batch completes and a WHERE clause removes them.
- D. Incorrect: the `guardrails` option defaults to FALSE, so a call without options performs no Cortex Guard filtering. TRY_COMPLETE only changes error handling, not safety filtering.
Checkpoint 2 of 10· Check yourself
An auditor wants a span-level trace of a SQL pipeline that calls AI_COMPLETE on customer emails. Where will they find it?
Built-in AI functions record only credits, tokens, function name and model in Account Usage. If you need traces, wrap the pipeline as a custom app instrumented with TruLens.
“Built-in functions such as AI_COMPLETE and AI_CLASSIFY do not write traces to the observability event table.”Source: docs.snowflake.com
2.LLM-as-a-judge evaluations
Traces tell you what happened. Evaluations tell you whether it was any good. AI Observability computes metrics from the application's inputs, its LLM-generated outputs and any intermediate data, such as the passages a RAG application retrieved. A ground-truth dataset can also be used. In the LLM-as-a-judge approach, a second LLM reads that information and produces a score between 0 and 1, along with an explanation. You can choose any LLM available in Cortex AI as the judge. If you don't choose one, llama3.1-70b is used.
Two documented metrics matter for data integrity. Context Relevance asks whether the retrieved context is relevant to the user's query. Groundedness asks whether the response is actually supported by that context. A low groundedness score means the model is saying things its sources don't support. The exam guide also mentions judging for bias and toxicity, but the sources for this lesson don't define metrics with those names. Cortex Agents and Cortex Analyst also support batch evaluations on a dataset, before or after deployment.
Checkpoint 3 of 10· Exam question
An analyst runs COMPLETE and TRY_COMPLETE on the same prompt with `guardrails` enabled, and Cortex Guard filters the model response. Which statement describes what each function does?
Correct answer: D — COMPLETE raises an error for the filtered response, while TRY_COMPLETE returns NULL for that row so the query keeps running without an exception.
- A. Incorrect: this reverses the behavior. NULL-on-failure is the defining trait of TRY_COMPLETE, and COMPLETE is the one that raises.
- B. Incorrect: neither function substitutes a placeholder string. A filtered response is surfaced as an error or as NULL depending on the function.
- C. Incorrect: Cortex Guard withholds the response rather than annotating it. No `filtered` flag exists that lets the caller receive the unsafe text.
- D. Correct: COMPLETE raises errors when it cannot complete the operation, while TRY_COMPLETE converts the failure, including a Cortex Guard filter, into NULL. That makes TRY_COMPLETE the fault-tolerant choice for batch work.
Checkpoint 4 of 10· Check yourself
What does an LLM-as-a-judge metric produce for each evaluated output?
The judge LLM scores the output from 0 to 1 and explains the score, based on the inputs, outputs and intermediate information it is given.
“an LLM is used to generate a score (between 0 and 1) with an explanation for the application’s output”Source: docs.snowflake.com
Sources3
3.Monitoring security and data-quality signals
For ongoing monitoring, use the Account Usage views rather than raw traces. The most direct security metric is the guardrail signal. CORTEX_AI_GUARDRAILS_USAGE_HISTORY records every guardrail scan, and rows with GUARDRAILS_SIGNAL = TRUE are requests flagged for possible prompt injection. You can audit them by user, agentic source or role.
SELECT *
FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_AI_GUARDRAILS_USAGE_HISTORY
WHERE GUARDRAILS_SIGNAL = TRUE
AND USAGE_TIME >= DATEADD('hour', -72, CURRENT_TIMESTAMP())
LIMIT 100;Review this view regularly in both directions. Legitimate prompts are sometimes flagged, and patterns in those false positives are worth investigating. For volume and cost, use the usage views: request IDs, user and agent identifiers, token counts and models. Don't sum the event table. Trace delivery is best effort, so totals calculated from AI_OBSERVABILITY_EVENTS can under-report.
Checkpoint 5 of 10· Check yourself
A security team builds a dashboard of total Cortex Agent consumption per user by counting rows in AI_OBSERVABILITY_EVENTS. What is wrong with this?
Snowflake recommends basing spend reporting on the Account Usage views, because traces may be missing.
“Trace delivery is best effort, so totals computed from the event table can under-report consumption.”Source: docs.snowflake.com
Sources4
4.Securely configuring Cortex Analyst
Cortex Analyst turns natural-language questions into SQL. It uses a semantic model, which can be a semantic view (the recommended approach) or a legacy YAML file on a stage. The model's metadata is used only to generate SQL. That SQL then runs in your own warehouse under RBAC, so it is bound by the caller's existing access controls. Semantic views add their own controls: privilege management, and access modifiers that mark facts and metrics as public or private.
The privileges stack up. The caller needs SELECT on the tables in the model, USAGE on any Cortex Search services it references, and READ or WRITE on the stage if the model is a file. To restrict which users can reach a given model, keep the YAML in a stage and control access to that stage.
The main hardening step is about defaults. CORTEX_USER is granted to PUBLIC by default, which gives every user all Covered AI features. To allow Analyst only, revoke CORTEX_USER and grant SNOWFLAKE.CORTEX_ANALYST_USER to a custom role, then assign that role to users. Database roles can't be granted directly to users.
Checkpoint 6 of 10· Fill the gap
Before giving users Analyst-only access, ACCOUNTADMIN must remove the broader role. Complete the command.
REVOKE DATABASE ROLE SNOWFLAKE. ? FROM ROLE analyst;CORTEX_USER grants every Covered AI feature. Revoking it and granting CORTEX_ANALYST_USER limits users to Cortex Analyst.
Source: docs.snowflake.comSources5
5.Auditing Analyst questions and generated SQL
Each Analyst request is logged with the user who asked, the question, the generated SQL, any errors or warnings, the request and response bodies, and other metadata. Logs appear after a lag of about 1 to 2 minutes. Direct Analyst requests are stored in SNOWFLAKE.LOCAL.CORTEX_ANALYST_REQUESTS_RAW, not in the shared observability event table. You can browse them in the semantic view's Monitoring tab in Snowsight, or query them with a table function.
SELECT * FROM TABLE(
SNOWFLAKE.LOCAL.CORTEX_ANALYST_REQUESTS(
'<semantic_model_or_view_type>',
'<semantic_model_or_view_name>'
)
);The logs are protected too. The function checks access before returning anything. To read the logs, you need SELECT on the referenced tables, plus MONITOR or OWNERSHIP on the semantic view, or WRITE on the stage for a staged YAML model.
Checkpoint 7 of 10· Exam question
A security administrator finds that every user in the account can call Cortex LLM functions because the CORTEX_USER database role was inherited through PUBLIC. Only members of `ANALYST_ROLE` should be allowed to call them. Which TWO actions achieve this?(Select 2)
Correct answers: A, D — Run `GRANT DATABASE ROLE SNOWFLAKE.CORTEX_USER TO ROLE ANALYST_ROLE` so that only that role carries the privilege to call the functions.; Run `REVOKE DATABASE ROLE SNOWFLAKE.CORTEX_USER FROM ROLE PUBLIC` so that the default account-wide access is removed.
- A. Correct: granting the CORTEX_USER database role to a specific custom role gives the intended group access. Combined with the revoke, it restricts use to that role.
- B. Incorrect: a resource monitor limits credit consumption on a warehouse and does not authorize or deny Cortex function calls by role.
- C. Incorrect: usage on the SNOWFLAKE database does not confer Cortex access. The CORTEX_USER database role carries that privilege.
- D. Correct: removing the role from PUBLIC ends the default access that every user inherits. Without this step, the targeted grant would not restrict anyone.
- E. Incorrect: the cross-region parameter controls whether requests may be processed in another region, not which roles may call functions.
Checkpoint 8 of 10· Check yourself
A reviewer has SELECT on every table in a semantic view but cannot see its Cortex Analyst logs. What are they most likely missing?
Reading the logs for a semantic view requires MONITOR or OWNERSHIP on that view in addition to SELECT on the referenced tables.
“MONITOR or OWNERSHIP on the semantic view (when using semantic views)”Source: docs.snowflake.com
Sources6
6.Governing Cortex Agent orchestration and tools
A Cortex Agent is a schema-level object that bundles a model, tools and instructions. It runs a loop: plan, use tools, reflect. You shape this orchestration with natural-language planning and response instructions. The tools set the security boundary: Cortex Analyst over semantic views, Cortex Search, sandboxed code execution, custom stored procedures and UDFs, skills, MCP connectors and web search. Each tool reads data in its own execution context. If the caller's role can't use some of the configured tools, the run can continue with the ones it can access.
When you automate governance workflows with agents, three controls matter. First, choose tools deliberately: web search and MCP connectors send requests over the public internet, while most other tools stay inside Snowflake. Second, create the agent as a secure agent if non-owner roles shouldn't see its specification. Third, review answers before serving them, because Snowflake doesn't guarantee LLM accuracy. Account-level Cortex AI Guardrails cover agents and scan tool outputs for indirect prompt injection.
Checkpoint 9 of 10· Check yourself
A governance agent must not send any request outside Snowflake. Which configured tool breaks that requirement?
MCP connectors and web search reach external services over the public internet. Analyst, Search and custom tools run inside Snowflake.
“Tools that reach external services, such as web search and MCP connectors, send requests over the public internet.”Source: docs.snowflake.com
Sources2
7.Horizon Catalog and AI-assisted security analysis
The exam guide asks you to use Copilot for Snowflake Horizon Catalog to analyse and audit security. None of the sources for this lesson describes a feature by that name, so this lesson can't teach its configuration or behaviour. Here is what the sources do say. Horizon Catalog includes AI guardrails and AI governance among its governance capabilities, alongside sensitive data protection and lineage. Cortex AI Guardrails, covered above, are part of Horizon Catalog. Separately, Horizon Context lets CoCo answer natural-language questions about lineage, such as which downstream objects would break after a change, ranked by risk. That is a related capability, not the Copilot named in the guide.
Checkpoint 10 of 10· Check yourself
According to the sources, which security control for AI applications is part of Snowflake Horizon Catalog?
Cortex AI Guardrails are part of Horizon Catalog and protect CoCo, Snowflake CoWork and Cortex Agents against prompt injection and jailbreak attacks.
“Cortex AI Guardrails, part of the Snowflake Horizon Catalog, provide run-time protection against prompt injection and jailbreak attacks”Source: docs.snowflake.com
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.Direct Cortex Analyst requests are logged in AI_OBSERVABILITY_EVENTS with agent traces.Why is that wrong?
Direct Analyst telemetry is stored in a separate LOCAL table. Analyst steps appear in AI_OBSERVABILITY_EVENTS only when an agent calls Analyst as a tool.
Covered in Auditing Analyst questions and generated SQL
2.Granting CORTEX_ANALYST_USER to a role is enough to restrict users to Cortex Analyst only.Why is that wrong?
CORTEX_USER is granted to PUBLIC by default and gives every user all Covered AI features. It must be revoked for the narrower role to have any effect.
Covered in Securely configuring Cortex Analyst
3.Every Cortex call, including AI_COMPLETE in SQL, produces a trace you can inspect in the observability event table.Why is that wrong?
Built-in functions and REST API inference write no traces. They leave only usage records in Account Usage.
Covered in Where AI traces live
Practise it for real
Enable account-level Cortex AI Guardrails and audit what they flag
1.As ACCOUNTADMIN, run ALTER ACCOUNT SET AI_SETTINGS with guardrails.advanced_prompt_injection enabled: true.
Why: Cortex AI Guardrails are configured centrally through the AI_SETTINGS account parameter.
You should see: The statement succeeds if the account meets the prerequisite that cross-region inference is enabled.
2.Run SHOW PARAMETERS LIKE 'AI_SETTINGS' IN ACCOUNT.
Why: This confirms which guardrail configuration is in effect.
You should see: The AI_SETTINGS value shows the guardrails block you set.
3.Send some prompts to a Cortex Agent, then query SNOWFLAKE.ACCOUNT_USAGE.CORTEX_AI_GUARDRAILS_USAGE_HISTORY WHERE GUARDRAILS_SIGNAL = TRUE.
Why: The view records every guardrail scan and identifies flagged requests by user, source and role.
You should see: Scan rows for the agent requests; any flagged ones have GUARDRAILS_SIGNAL = TRUE.
Stuck? Get a nudge
If ALTER ACCOUNT fails, check CORTEX_ENABLED_CROSS_REGION before anything else.
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“Cortex Agents deployed through the Agent API or Snowflake CoWork log conversation threads, turns, and execution spans automatically into SNOWFLAKE.LOCAL.AI_OBSERVABILITY_EVENTS.”
↩︎ Where AI traces live“Snowflake registers each TruLens application as an External Agent object. That object stores metadata only.”
↩︎ Where AI traces live“Cortex Analyst stores direct Analyst request and response telemetry in SNOWFLAKE.LOCAL.CORTEX_ANALYST_REQUESTS_RAW, not in the shared AI observability event table.”
↩︎ Exam trap 1“REST API inference does not write to the observability event table.”
↩︎ Exam trap 3“Built-in functions such as AI_COMPLETE and AI_CLASSIFY do not write traces to the observability event table.”
↩︎ Checkpoint“Trace delivery is best effort, so totals computed from the event table can under-report consumption.”
↩︎ Checkpoint - 2.
“The agent emits events throughout the run that surface its reasoning, tool calls, and reflections.”
↩︎ Where AI traces live“Secure agents hide the specification from non-owner roles.”
↩︎ Governing Cortex Agent orchestration and tools“You should review all answers from the Agents API before serving them to your users.”
↩︎ Governing Cortex Agent orchestration and tools“Tools that reach external services, such as web search and MCP connectors, send requests over the public internet.”
↩︎ Checkpoint - 3.
“These metrics are computed using specific inputs to the application, LLM-generated outputs, and any intermediate information (such as retrieved results from a RAG application).”
↩︎ LLM-as-a-judge evaluations“Groundedness determines if the generated response is supported by and grounded in the retrieved context”
↩︎ LLM-as-a-judge evaluations“If no LLM judge is specified, llama3.1-70b is used as the default judge.”
↩︎ Prediction“an LLM is used to generate a score (between 0 and 1) with an explanation for the application’s output”
↩︎ Checkpoint - 4.
“Review which requests were flagged for possible prompt injection (GUARDRAILS_SIGNAL = TRUE)”
↩︎ Monitoring security and data-quality signals“Cortex AI Guardrails, part of the Snowflake Horizon Catalog, provide run-time protection against prompt injection and jailbreak attacks”
↩︎ Checkpoint - 5.
“ensuring that SQL queries generated and executed adhere to all established access controls.”
↩︎ Securely configuring Cortex Analyst“CORTEX_USER provides access to all Covered AI features, while CORTEX_ANALYST_USER provides access only to Cortex Analyst.”
↩︎ Securely configuring Cortex Analyst“Access modifiers: Mark facts and metrics as public or private to control visibility”
↩︎ Securely configuring Cortex Analyst“By default, the CORTEX_USER role is granted to the PUBLIC role.”
↩︎ Exam trap 2 - 6.https://docs.snowflake.com/en/user-guide/snowflake-cortex/cortex-analyst/admin-observabilityOfficial docs
“This table function performs access control checks to ensure that the caller has required privileges to access the request data.”
↩︎ Auditing Analyst questions and generated SQL“MONITOR or OWNERSHIP on the semantic view (when using semantic views)”
↩︎ Checkpoint - 7.
“trust every answer with enterprise-grade governance: sensitive data protection, data quality, end-to-end lineage, AI guardrails, and AI governance.”
↩︎ Horizon Catalog and AI-assisted security analysis - 8.
“CoCo reads the same lineage that the catalog exposes and reports the downstream objects, ranked by risk”
↩︎ Horizon Catalog and AI-assisted security analysis