CertSafari
    Snowflake SnowPro Specialty: Gen AI (GES-C02)· Lessons

    Domain 3 · Lesson 11/15

    Snowflake AI Observability with TruLens: tracing, event tables and evaluations

    Use Snowflake AI observability tools.

    21 min read
    7.25% of exam
    4 sources
    Published 21 Oct 2026
    Docs as of 4 Oct 2026

    What you will be able to do

    • Explain what Snowflake AI Observability does for a custom generative AI application and where it stops
    • Instrument app functions with TruLens @instrument() and choose RECORD_ROOT, RETRIEVAL or GENERATION span types
    • Register an instrumented app as an External Agent and say what that object does and does not store
    • Query traces and run logs in SNOWFLAKE.LOCAL.AI_OBSERVABILITY_EVENTS and explain who can read them
    • Create, invoke and score evaluation runs with LLM-as-a-judge and custom metrics, then compare runs
    • Grant the privileges a role needs to register apps, execute runs and compute judge metrics

    Key concept

    External Agent plus the AI observability event table — A TruLens-instrumented application is registered in Snowflake as an External Agent object. That object only holds metadata such as name and version. Every trace, span and evaluation score the app produces goes into the shared SNOWFLAKE.LOCAL.AI_OBSERVABILITY_EVENTS table.

    1.What AI Observability does, and for which apps

    Snowflake AI Observability shows you what a generative AI application did and how good its answers were. It is a set of features across Cortex products that answers questions like these: what happened in one specific production request, how well the app performs on test data, and what a request cost. For an application you build yourself, it does two jobs. It traces each execution step by step, and it evaluates output quality with LLM-as-a-judge metrics. Both jobs are built on TruLens, an open source observability SDK that Snowflake integrates.

    Which tool you use depends on who built the app. Native features handle their own telemetry. A Cortex Agent deployed through the Agent API logs threads, turns and spans automatically, so it needs no TruLens. TruLens is for applications you own end to end. Examples are a RAG pipeline that combines Cortex Search with AI_COMPLETE, a graph of Cortex Agents, or a custom retrieval chain. The app can run on Snowpark Container Services, on another cloud, or on-premises, and it still streams its traces into your Snowflake account.

    Traces and scores are evidence that you read and act on. They are not a control mechanism, and they are not a billing ledger. Trace delivery is best effort, so totals added up from trace data can under-report consumption. Spend reporting and caps belong to the Account Usage views and budgets (cost monitoring), not to the observability event table.

    Checkpoint 1 of 10· Check yourself

    Your team needs step-by-step traces and LLM-judge quality scores. Which workload is the one where you should instrument and register the app with TruLens?

    Checkpoint 2 of 10· Exam question

    What is the primary purpose of Snowflake AI Observability?

    Sources12

    2.Instrumenting app functions with TruLens span types

    Instrumentation begins with three Python packages at version 2.1.2 or later: trulens-core, trulens-connectors-snowflake and trulens-providers-cortex. You then put the @instrument() decorator on each function you want traced. A decorated function becomes a span that records its inputs, its outputs and its latency.

    A span type tells Snowflake what role the function plays in the app. There are three:

    • RECORD_ROOT marks the main entry point. One call to it is one record: the whole invocation, from the user's question to the final answer.
    • RETRIEVAL marks a retrieval or search function.
    • GENERATION marks an LLM inference call.
    Tagging the retrieval, generation and entry-point functions of a RAG app with span typespython
    @instrument(span_type=SpanAttributes.SpanType.RETRIEVAL)
    def retrieve_context(self, query: str) -> list:
        return self.retrieve(query)
    
    @instrument(span_type=SpanAttributes.SpanType.GENERATION)
    def generate_completion(self, query: str, context_str: list) -> str:
        return response
    
    @instrument(span_type=SpanAttributes.SpanType.RECORD_ROOT)
    def answer_query(self, query: str) -> str:
        context_str = self.retrieve_context(query)
        return self.generate_completion(query, context_str)

    Span types are what make metrics possible, not just tidier traces. A metric is computed from span attributes, and each span type carries a fixed set of them. You map a function parameter, or its return value, to an attribute through the attributes argument of @instrument(). For example, the query parameter can map to RETRIEVAL.QUERY_TEXT and return to RETRIEVAL.RETRIEVED_CONTEXTS. The attributes argument also accepts a lambda that receives ret, exception, *args and **kwargs, so you can compute an attribute from the call, such as pulling just the text field out of each returned document.

    Span types, the attributes they carry, and the metrics those attributes feed
    Span typeMarksSupported attributesMetrics that depend on it
    RECORD_ROOTMain entry point: overall input and output of one invocationINPUT, OUTPUT, GROUND_TRUTH_OUTPUTanswer_relevance, correctness, coherence; OUTPUT also feeds groundedness
    RETRIEVALRetrieval or search functionQUERY_TEXT, RETRIEVED_CONTEXTScontext_relevance; RETRIEVED_CONTEXTS also feeds groundedness
    GENERATIONLLM inference callNoneNone; the span adds trace readability

    Apps built on a common framework don't need hand-placed decorators. TruLens provides wrappers that instrument them automatically: TruChain for LangChain and LCEL chains, TruGraph for LangGraph, and TruLlama for LlamaIndex query engines and retrievers.

    Checkpoint 3 of 10· Fill the gap

    In this RAG app, answer_query takes the user's question and returns the final answer. Which span type completes its decorator so that each call counts as one record, and so that answer_relevance, correctness and coherence can be computed?

    @instrument(span_type=SpanAttributes.SpanType.RETRIEVAL)
    def retrieve_context(self, query: str) -> list:
        return self.retrieve(query)
    
    @instrument(span_type=SpanAttributes.SpanType.GENERATION)
    def generate_completion(self, query: str, context_str: list) -> str:
        return response
    
    @instrument(span_type=SpanAttributes.SpanType. ? )
    def answer_query(self, query: str) -> str:
        context_str = self.retrieve_context(query)
        return self.generate_completion(query, context_str)

    Checkpoint 4 of 10· Exam question

    An engineer instruments the top-level entry-point function of a custom RAG application with TruLens and tags its span type as RECORD_ROOT. What is the purpose of the RECORD_ROOT span type?

    Sources32

    3.Registering the app as an External Agent

    Instrumented code sends nothing to Snowflake until you register the app. You register it with TruApp, and registration does two things. It lets TruLens write traces to the event table, and it creates an External Agent object, which you govern like any other schema object. The framework wrappers register the app automatically when they wrap it.

    TruApp registration signaturepython
    tru_app = TruApp(
        app: Any,
        app_name: str,
        app_version: str,
        connector: SnowflakeConnector,
        main_method: callable  # for example app.query
    )

    The parameters:

    • app_name is the name Snowflake stores for the application.
    • app_version is the label you later compare experiments by.
    • connector is a SnowflakeConnector that manages the Snowpark session.
    • main_method is optional when the entry point already carries RECORD_ROOT instrumentation.

    Traces are exported only for calls made while the recorder is active.

    An External Agent object is small. It holds only the application name, version and run name. It contains no code, prompts, traces or scores. Two consequences matter in practice. First, External Agents share a namespace with model objects in the same schema, so registration fails if a model in that schema already uses the name. Second, the role that registers needs CREATE EXTERNAL AGENT on the schema, and changing or dropping the object later with SQL requires OWNERSHIP.

    Checkpoint 5 of 10· Check yourself

    A data engineer calls TruApp with app_name="SUPPORT_RAG" to register an app in schema APPS.ML. The app's entry point is decorated with RECORD_ROOT, so main_method is omitted. The role has CREATE EXTERNAL AGENT on APPS.ML, and no External Agent named SUPPORT_RAG exists there. Registration still fails. What is the most likely cause?

    Sources34

    4.The AI observability event table, logs and access

    SNOWFLAKE.LOCAL.AI_OBSERVABILITY_EVENTS is a Snowflake-managed table, and TruLens apps are only one of the features that write to it. You don't create it or configure it. Its rows can't be modified after ingestion. To read it, you call the table function SNOWFLAKE.LOCAL.GET_AI_OBSERVABILITY_EVENTS with the database, schema and External Agent name and the agent type 'EXTERNAL AGENT'. Results use the event table's column layout. To narrow them to one evaluation run, filter on RECORD_ATTRIBUTES, for example by run name.

    Which AI features write traces to AI_OBSERVABILITY_EVENTS
    FeatureWhere its telemetry goes
    TruLens External AgentsAI_OBSERVABILITY_EVENTS (traces and evaluation scores)
    Cortex AgentsAI_OBSERVABILITY_EVENTS, logged automatically
    Cortex SearchAI_OBSERVABILITY_EVENTS, one row per request, only when REQUEST_LOGGING is enabled
    Direct Cortex Analyst requestsSNOWFLAKE.LOCAL.CORTEX_ANALYST_REQUESTS_RAW, not the shared event table
    Built-in functions such as AI_COMPLETE, and the REST APINo traces; usage only in Account Usage views

    Run logs are separate from traces. Warnings and errors raised during a TruLens run come from a second function, GET_AI_OBSERVABILITY_LOGS, which takes the same arguments. Only two log fields are guaranteed to stay stable: record:"severity_text" and the run name in record_attributes. Other fields may change, so filter on those two.

    Pulling WARN and ERROR log entries for one evaluation runsql
    SELECT *
      FROM TABLE(SNOWFLAKE.LOCAL.GET_AI_OBSERVABILITY_LOGS(
        '<database_name>',
        '<schema_name>',
        '<external_agent_name>',
        'EXTERNAL AGENT'
      ))
      WHERE record:"severity_text" IN ('ERROR', 'WARN')
        AND record_attributes:"snow.ai.observability.run.name" = '<run_name>';

    Access to this data has three layers:

    • Calling the function. For an External Agent, USAGE on the agent is enough to call the table functions. MONITOR does not apply.
    • Reading or deleting rows directly. The SNOWFLAKE.AI_OBSERVABILITY_READER application role gives read-only access to the table. SNOWFLAKE.AI_OBSERVABILITY_ADMIN can delete rows.
    • Seeing sensitive content. The account privilege READ UNREDACTED AI OBSERVABILITY EVENTS TABLE decides whether a role sees unredacted tool inputs, outputs and conversation text in Snowsight and in the UDTFs. Without it, the role still sees metadata such as tool names, latency and token usage. The privilege does not change how evaluation runs execute or how their scores appear.

    Checkpoint 6 of 10· Fill the gap

    Several records in a TruLens evaluation run failed, and you want only the warnings and errors raised during that run. Which table function completes this query?

    SELECT *
      FROM TABLE(SNOWFLAKE.LOCAL. ? (
        '<database_name>',
        '<schema_name>',
        '<external_agent_name>',
        'EXTERNAL AGENT'
      ))
      WHERE record:"severity_text" IN ('ERROR', 'WARN')
        AND record_attributes:"snow.ai.observability.run.name" = '<run_name>';

    Checkpoint 7 of 10· Exam question

    Which combination correctly identifies where AI Observability trace and evaluation data are stored, and how an engineer queries them directly in SQL?

    Sources312

    5.Datasets and LLM-as-a-judge evaluation metrics

    Evaluation starts with a dataset: a set of test inputs, and optionally the answers you expect (the ground truth). You can supply it as a Snowflake table or as a pandas dataframe. Each column must map to a reserved attribute:

    • RECORD_ROOT.INPUT: the prompt.
    • RECORD_ROOT.INPUT_ID: an identifier for the prompt, generated automatically if you leave it out.
    • RETRIEVAL.QUERY_TEXT: the user query in a RAG app.
    • RECORD_ROOT.GROUND_TRUTH_OUTPUT: the expected response.

    Your instrumented functions supply the output attributes RETRIEVAL.RETRIEVED_CONTEXTS and RECORD_ROOT.OUTPUT.

    Scoring uses the LLM-as-a-judge approach. A judge LLM gives each output a score between 0 and 1, along with an explanation of the score. Any LLM available in Cortex AI can be the judge, and llama3.1-70b is used when you don't name one. Each metric checks one specific relationship, so it needs specific attributes. If you leave an attribute unmapped, the metric has nothing to score.

    Server-side metrics and the attributes each one requires
    MetricWhat the judge decidesRequired attributes
    context_relevanceIs the retrieved context relevant to the query?RETRIEVAL.QUERY_TEXT, RETRIEVAL.RETRIEVED_CONTEXTS
    groundednessIs the response supported by the retrieved context? (uses chain-of-thought reasoning)RETRIEVAL.RETRIEVED_CONTEXTS, RECORD_ROOT.OUTPUT
    answer_relevanceIs the response relevant to the query? (no ground truth)RECORD_ROOT.INPUT, RECORD_ROOT.OUTPUT
    correctnessHow closely does the response align with the ground truth?RECORD_ROOT.INPUT, RECORD_ROOT.GROUND_TRUTH_OUTPUT, RECORD_ROOT.OUTPUT
    coherenceIs the response free of logical gaps and contradictions?RECORD_ROOT.OUTPUT

    Cost and latency come from the traces, not from a judge. Cost is calculated for each Cortex LLM call from the prompt_tokens and completion_tokens that AI_COMPLETE returns, and you can see it per call in the trace. Latency is measured for each instrumented function and then added up for the whole application on each input. Each run also reports an average latency across all inputs.

    Snowflake computes the metrics above on the server when you request them by name. Custom metrics run on the client: you wrap any Python function, or any TruLens feedback function, in a Metric object and pass it in the same list as the named metrics. A feedback function such as provider.relevance_with_cot_reasons can be tuned to your domain with criteria, additional_instructions, scored examples, and a custom score range set by min_score_val and max_score_val.

    A client-side custom metric built from a plain Python functionpython
    word_count = Metric(
        name="word_count",
        implementation=lambda text: len(text.split()),
        selectors={"text": Selector.select_record_output()},
    )

    Checkpoint 8 of 10· Match them up

    Match each server-side evaluation metric to the question its LLM judge answers.

    Tap a term, then the definition that fits it.

    Sources42

    6.Creating, invoking and comparing runs

    A run is one batch evaluation of one application version against one dataset. Building a gen AI app means trying different LLMs, prompts and inference parameters. Each combination you try is a separate version, and each run measures one of them. You create a run with RunConfig and tru_app.add_run(). The dataset_spec maps attribute names to dataset columns, and llm_judge_name picks the judge model.

    Configuring a run against a dataframe with a non-default judgepython
    run_config = RunConfig(
        run_name=run_name,
        description="desc",
        label="custom tag useful for grouping comparable runs",
        source_type="DATAFRAME",
        dataset_name="My test dataframe name",
        dataset_spec={
            "RETRIEVAL.QUERY_TEXT": "user_query_field",
            "RECORD_ROOT.INPUT": "user_query_field",
            "RECORD_ROOT.GROUND_TRUTH_OUTPUT": "golden_answer_field",
        },
        llm_judge_name="mistral-large2"
    )
    
    run = tru_app.add_run(run_config=run_config)

    A run has four phases, and the first two happen without any scoring. The phases are creation, invocation, computation and visualization:

    1. Invocation. run.start() reads each input, calls your app, and stores the traces. Use run.start(input_df=...) for a dataframe source. The call blocks until ingestion completes or times out.
    2. Computation. Scoring is a separate, asynchronous call: run.compute_metrics([...]). You can call it after invocation reaches INVOCATION_COMPLETED or INVOCATION_PARTIALLY_COMPLETED.

    You can call compute_metrics again later with different metrics, but you can't recompute a metric the run already has.

    Run statuses in lifecycle order
    StatusMeaning
    CREATEDCreated but not started
    INVOCATION_IN_PROGRESSGenerating outputs and traces
    INVOCATION_COMPLETEDAll outputs and traces created; metrics not yet computed
    INVOCATION_PARTIALLY_COMPLETEDSome app invocations or trace generation failed
    COMPUTATION_IN_PROGRESSMetric computation running
    COMPLETEDMetric computation done
    PARTIALLY_COMPLETEDSome metric computations failed
    CANCELLEDRun cancelled

    Comparison is the reason runs exist. Run the same dataset against version 1 and version 2. Then, in Snowsight under AI & ML » Evaluations, select the External Agent, select both runs, and choose Compare. You see differences in aggregate scores and record by record, which shows whether a prompt change helped and which version to deploy. Selecting a single record opens its traces, latency, span detail and the judge's explanation. A label groups runs that are meant to be compared, so you can filter for them.

    Runs also cover production monitoring and clean-up:

    • Live runs. Put @trace_with_run(app=tru_app, run_name=...) on the function that calls your app, typically a web or API handler. Every production call is then traced into that run.
    • Clean-up. run.cancel() stops a run. run.delete() removes only the run's metadata, and its traces stay in AI_OBSERVABILITY_EVENTS.

    Checkpoint 9 of 10· Put it in order

    Version 1 of a RAG app already has a scored run. You now want to evaluate version 2 on the same dataset and decide which version to deploy. Put the steps for version 2 in order.

    1. 1.After the status reaches INVOCATION_COMPLETED, call run.compute_metrics() with the metrics to score
    2. 2.Call run.start() so the app is invoked on each input and its outputs and traces are stored
    3. 3.Create a run for version 2 with RunConfig and tru_app.add_run(), pointing dataset_spec at the same test dataset
    4. 4.In Snowsight under AI & ML » Evaluations, select the version 1 and version 2 runs and choose Compare

    Sources24

    7.Privileges for registering apps and running evaluations

    Evaluation calls a Cortex model, so the role that evaluates must pass the Cortex gates. Access to Cortex functions depends on two things: the account-level USE AI FUNCTIONS privilege and the SNOWFLAKE.CORTEX_USER database role (The two gates: USE AI FUNCTIONS privilege and the SNOWFLAKE.CORTEX_USER database role). AI Observability's judge is AI_COMPLETE, which means the role computing metrics must be able to call AI_COMPLETE. That role needs CORTEX_USER to use AI Observability at all. It also needs USE AI FUNCTIONS, or the per-function USE AI FUNCTION AI_COMPLETE privilege. PUBLIC holds USE AI FUNCTIONS by default. If your account revoked it from PUBLIC, metric computation fails until you grant it explicitly.

    Runs need further privileges because they execute as tasks:

    • USAGE on the External Agent.
    • CREATE TASK on the schema where the app is registered.
    • The global EXECUTE TASK privilege, to run the task that invokes the application.

    Registration on its own needs only CREATE EXTERNAL AGENT on the schema.

    One role carrying every privilege the observability workflow needssql
    USE ROLE ACCOUNTADMIN;
    
    CREATE ROLE observability_user_role;
    
    GRANT DATABASE ROLE SNOWFLAKE.CORTEX_USER TO ROLE observability_user_role;
    
    GRANT CREATE EXTERNAL AGENT ON SCHEMA app_schema TO ROLE observability_user_role;
    
    GRANT CREATE TASK ON SCHEMA app_schema TO ROLE observability_user_role;
    
    GRANT EXECUTE TASK ON ACCOUNT TO ROLE observability_user_role;
    
    GRANT USE AI FUNCTIONS ON ACCOUNT TO ROLE observability_user_role;
    
    GRANT ROLE observability_user_role TO USER some_user;

    Checkpoint 10 of 10· Fill the gap

    An administrator is building one role that can register a TruLens app, execute runs as tasks and have the AI_COMPLETE judge compute metrics. Which database role completes the first grant?

    USE ROLE ACCOUNTADMIN;
    
    CREATE ROLE observability_user_role;
    
    GRANT DATABASE ROLE SNOWFLAKE. ?  TO ROLE observability_user_role;
    
    GRANT CREATE EXTERNAL AGENT ON SCHEMA app_schema TO ROLE observability_user_role;
    
    GRANT CREATE TASK ON SCHEMA app_schema TO ROLE observability_user_role;
    
    GRANT EXECUTE TASK ON ACCOUNT TO ROLE observability_user_role;
    
    GRANT USE AI FUNCTIONS ON ACCOUNT TO ROLE observability_user_role;
    
    GRANT ROLE observability_user_role TO USER some_user;

    Sources4

    Sources

    (4)

    Every claim above is drawn from one of these pages, quoted as it was written on the date shown.

    1. 1.
      “AI observability is a collection of features inside Cortex products that help you answer questions about AI workloads in your account”
      ↩︎ What AI Observability does, and for which apps
      “Custom AI applications you host through Snowflake products, or even outside of Snowflake, can stream telemetry information into your account with TruLens.”
      ↩︎ What AI Observability does, and for which apps
      “Base spend reporting on the usage views rather than on AI_OBSERVABILITY_EVENTS.”
      ↩︎ What AI Observability does, and for which apps
      “Cortex Agents deployed through the Agent API or Snowflake CoWork log conversation threads, turns, and execution spans automatically into SNOWFLAKE.LOCAL.AI_OBSERVABILITY_EVENTS.”
      ↩︎ The AI observability event table, logs and access
      “Built-in functions such as AI_COMPLETE and AI_CLASSIFY do not write traces to the observability event table.”
      ↩︎ The AI observability event table, logs and access
      “Snowflake registers each TruLens application as an External Agent object. That object stores metadata only.”
      ↩︎ Key concept
      “Trace delivery is best effort, so totals computed from the event table can under-report consumption.”
      ↩︎ Exam trap 1
    2. 2.
      “Evaluations invoke your app against a dataset, store traces, and compute LLM-as-a-judge metrics (such as context relevance, groundedness, and correctness).”
      ↩︎ What AI Observability does, and for which apps
      “RECORD_ROOT: INPUT, OUTPUT, GROUND_TRUTH_OUTPUT”
      ↩︎ Instrumenting app functions with TruLens span types
      “Filter on RECORD_ATTRIBUTES (for example run name) to scope results to one evaluation run.”
      ↩︎ The AI observability event table, logs and access
      “Client-side metrics can be any TruLens feedback function or any Python function.”
      ↩︎ Datasets and LLM-as-a-judge evaluation metrics
      “run.start() is blocking until invocation and ingestion complete or time out.”
      ↩︎ Creating, invoking and comparing runs
      “To compute server-side metrics after invocation status is INVOCATION_COMPLETED or INVOCATION_PARTIALLY_COMPLETED:”
      ↩︎ Creating, invoking and comparing runs
      “To compare runs that share a dataset, select multiple runs and choose Compare.”
      ↩︎ Creating, invoking and comparing runs
      “For online tracing (production monitoring), create a live run with the @trace_with_run decorator on the function that calls your app”
      ↩︎ Creating, invoking and comparing runs
    3. 3.
      “Install TruLens packages (version 2.1.2 or later): trulens-core, trulens-connectors-snowflake, trulens-providers-cortex.”
      ↩︎ Instrumenting app functions with TruLens span types
      “RECORD_ROOT: Main entry point for the application”
      ↩︎ Instrumenting app functions with TruLens span types
      “Register the application so TruLens can write traces to AI_OBSERVABILITY_EVENTS and create an External Agent object for governance”
      ↩︎ Registering the app as an External Agent
      “main_method (optional): Entry point for tracing (for example, app.answer_query). Not required if the entry point uses RECORD_ROOT instrumentation.”
      ↩︎ Registering the app as an External Agent
      “OWNERSHIP is required to modify or drop the External Agent with SQL.”
      ↩︎ Registering the app as an External Agent
      “The SNOWFLAKE.AI_OBSERVABILITY_READER application role grants read-only access to the table.”
      ↩︎ The AI observability event table, logs and access
      “Without the grant, metadata (tool names, latency, token usage, and similar fields) is still available.”
      ↩︎ The AI observability event table, logs and access
      “Snowflake registers each TruLens application as an External Agent object. That object stores metadata only (application name, version, run name).”
      ↩︎ Exam trap 2
      “External Agent objects share a namespace with model objects in the same schema.”
      ↩︎ Exam trap 3
      “When agent_type is EXTERNAL AGENT, USAGE on the External Agent is sufficient to call the function; MONITOR does not apply.”
      ↩︎ Exam trap 5
      “External Agent objects share a namespace with model objects in the same schema.”
      ↩︎ Checkpoint
      “Traces and evaluation results for TruLens applications are stored in SNOWFLAKE.LOCAL.AI_OBSERVABILITY_EVENTS.”
      ↩︎ Prediction
    4. 4.
      “To register an application, your role must have CREATE EXTERNAL AGENT privileges on the schema.”
      ↩︎ Registering the app as an External Agent
      “Each column in the dataset must be mapped to one of the following reserved attributes”
      ↩︎ Datasets and LLM-as-a-judge evaluation metrics
      “If no LLM judge is specified, llama3.1-70b is used as the default judge.”
      ↩︎ Datasets and LLM-as-a-judge evaluation metrics
      “Cost is calculated for each LLM invocation call that relies on Cortex LLMs based on the token usage information”
      ↩︎ Datasets and LLM-as-a-judge evaluation metrics
      “You can trigger multiple runs with the same dataset for different versions.”
      ↩︎ Creating, invoking and comparing runs
      “AI Observability uses the AI_COMPLETE function as the LLM judge, so the role that runs the computation must be able to call AI_COMPLETE.”
      ↩︎ Privileges for registering apps and running evaluations
      “To use AI Observability, your role must have the CORTEX_USER database role.”
      ↩︎ Privileges for registering apps and running evaluations
      “The EXECUTE TASK global privilege to execute the task that runs the application.”
      ↩︎ Privileges for registering apps and running evaluations
      “The run invocation completed with all outputs and traces created.”
      ↩︎ Exam trap 4
      “Correctness determines how aligned the generated response is with the ground truth.”
      ↩︎ Prediction
      “Groundedness determines if the generated response is supported by and grounded in the retrieved context from the retriever or the search service.”
      ↩︎ Checkpoint
      “Computation: After invocation, trigger computation by specifying metrics to be computed.”
      ↩︎ Checkpoint

    Ready to test yourself?

    Practise the 26 questions on this subdomain.

    Spotted a mistake, or was something unclear? Tell us.