Cortex AI Functions for Unstructured Data: Classification, Multimodal Extraction, Semantics and Cost

1.Cortex AI Functions: categorization and text analytics in SQL

Cortex AI Functions run LLM-powered analysis on text and images directly from SQL, and they are also available in Python. The models run inside the Snowflake service perimeter. To call them, a role needs the USE AI FUNCTIONS account-level privilege plus one of the CORTEX_USER or AI_FUNCTIONS_USER database roles.

Each function is built for a specific job, so you can drop it into a SELECT and use it as a step in a pipeline. For automated categorization, AI_CLASSIFY assigns text or images to categories you define, and AI_FILTER returns True or False so you can filter rows in WHERE or JOIN clauses. For text analytics, AI_SENTIMENT extracts sentiment, AI_SUMMARIZE condenses content, AI_TRANSLATE translates, AI_REDACT removes PII, and AI_EXTRACT pulls specific information out of text or files. AI_AGG and AI_SUMMARIZE_AGG work across many rows (for example, every support ticket in a table) and are not limited by the context window. AI_COMPLETE is the general-purpose function for most generative tasks.

These functions are tuned for throughput rather than latency, which suits batch pipelines that process large tables. For interactive, latency-sensitive applications, Snowflake points you to the REST APIs instead: Complete, Embed and Agents.

Sources1

2.Multimodal workflows: extracting data from documents, images, audio and video

The same functions also accept media files. They process files on internal or external stages and extract insights from text, visual and audio signals. The bridge from a file to a function is the FILE data type. TO_FILE creates a reference to a staged file that AI_COMPLETE and other file-accepting functions can read, and a stage's directory table supplies the RELATIVE_PATH of every file. One CREATE TABLE AS statement therefore turns a whole folder of media into a table of FILE objects:

-- Create video table using TO_FILE data type
CREATE OR REPLACE TABLE video_ads_table AS
SELECT
    TO_FILE('@video_ads', RELATIVE_PATH) AS video_file,
    RELATIVE_PATH
FROM DIRECTORY(@video_ads);

From that table, you choose a function according to the media type. When you call AI_COMPLETE with a response format, you can define the exact JSON schema to return, such as sentiment, detected brands, or a harmful-content flag, so the output arrives as structured data instead of free text. Note the release status: audio and video processing with AI_COMPLETE is in public preview, while the other multimodal capabilities are generally available.

AI_PARSE_DOCUMENT has two modes for documents: OCR mode extracts text, and LAYOUT mode extracts text with layout information. It can also pull out images embedded in a document. For speech, AI_TRANSCRIBE returns the text along with timestamps and speaker information. You can chain functions together: transcribe first, then pass the transcript to AI_COMPLETE, using PROMPT to build the prompt object.

SELECT  ? (TO_FILE('@podcast_videos_S3', 'podcast-interview.mp4'));

SELECT
    AI_COMPLETE('claude-sonnet-4-5',
        PROMPT('Return a list of any Retail Brands mentioned in this podcast {0}',
            TO_VARCHAR(transcription_results))) AS brands_identified
FROM podcast_video_transcription;

Sources21

3.Semantic analysis: embeddings, similarity and semantic views

Semantic analysis compares content by meaning, not exact words. AI_EMBED turns text or images into embedding vectors for similarity search, clustering and classification, and AI_SIMILARITY scores how similar two inputs are by their embeddings. For video, AI_MULTI_EMBED with the Twelve Labs Marengo 3 model converts each video into one or more 512-dimensional vectors that capture visual, audio and text meaning by segment. You store those vectors in a table and search them with SQL, for example to find every clip of someone riding a skateboard.

The exam guide also lists semantic views, which deal with meaning on the business side rather than in the files. A semantic view is a schema-level object that stores business concepts in the database: logical tables for entities such as customers or orders, relationships between them, and three kinds of definition. Facts are row-level attributes such as an individual sale amount. Metrics are aggregations such as Total Revenue. Dimensions are categorical attributes such as region or date. Defining "net revenue" once, with the correct aggregation, avoids dozens of inconsistent calculations spread across reports. You query a semantic view in a SELECT statement or use it from Cortex Agents, which read its definitions and generate SQL against the physical tables. You can create one with CREATE SEMANTIC VIEW, Semantic Studio in Workspaces, or a Snowsight wizard. Semantic views count as metadata.

Sources123

4.Managing Cortex AI Function cost

Cortex AI Functions are billed by the number of tokens processed, at a per-function rate in credits per million tokens. As a rule of thumb, one token is about four characters of text. Warehouse cost applies on top of that, which is why the MEDIUM ceiling matters. The billing details vary by function, and the differences are easy to test:

Some functions, including AI_CLASSIFY, AI_FILTER, AI_SENTIMENT and AI_TRANSLATE, add their own prompt to your input, so you are billed for more tokens than you supplied. AI_CLASSIFY also counts its labels, descriptions and examples as input tokens on every record, not once per call, so long category descriptions add up quickly on large tables. AI_COUNT_TOKENS lets you check prompt sizes before you spend.

To monitor spending, filter METERING_DAILY_HISTORY by service type, and use CORTEX_FUNCTIONS_USAGE_HISTORY for each function call or CORTEX_FUNCTIONS_QUERY_USAGE_HISTORY for each query. The per-query view has one row per model, so a query that calls two models shows two rows. Requests made through the REST API do not get this granular breakdown.

SELECT *
  FROM SNOWFLAKE.ACCOUNT_USAGE.METERING_DAILY_HISTORY
  WHERE SERVICE_TYPE='AI_SERVICES';

Sources4

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.Scaling up the warehouse speeds up Cortex AI Functions, the same way it speeds up ordinary queries.Why is that wrong?

   Snowflake recommends MEDIUM or smaller. A larger warehouse does not improve performance and only adds cost.

   Covered in Managing Cortex AI Function cost

2. 2.AI_PARSE_DOCUMENT is billed on input and output tokens, like AI_COMPLETE.Why is that wrong?

   AI_PARSE_DOCUMENT is billed by the number of document pages processed.

   Covered in Managing Cortex AI Function cost

3. 3.Every multimodal capability of AI_COMPLETE is generally available.Why is that wrong?

   Audio and video processing with AI_COMPLETE is in public preview. The other multimodal capabilities are GA.

   Covered in Multimodal workflows: extracting data from documents, images, audio and video

Practise it for real

Transcribe a staged video with SQL and measure what the transcription cost

1. 1.Create an internal stage with ENCRYPTION = (TYPE = 'SNOWFLAKE_SSE') and DIRECTORY = ( ENABLE = true ), then upload a short .mp4 file with PUT.

   Why: Server-side encryption keeps the files readable, and the directory table lists them for later steps.

   You should see: The stage exists and the PUT reports the file as uploaded.

2. 2.Refresh the directory metadata with ALTER STAGE ... REFRESH, then run SELECT * FROM DIRECTORY(@your_stage).

   Why: The directory table only shows files after its metadata is refreshed.

   You should see: One row with RELATIVE_PATH, SIZE, LAST_MODIFIED and FILE_URL for your video.

3. 3.Create a table of FILE objects with TO_FILE('@your_stage', RELATIVE_PATH) selected FROM DIRECTORY(@your_stage).

   Why: AI functions read staged media through FILE references.

   You should see: A table with one FILE value per staged video.

4. 4.Run AI_TRANSCRIBE on the FILE column.

   Why: This extracts the speech in the video as text.

   You should see: A JSON result that includes the audio_duration and text fields.

5. 5.Query SNOWFLAKE.ACCOUNT_USAGE.CORTEX_FUNCTIONS_QUERY_USAGE_HISTORY WHERE query_id = '<your query id>'.

   Why: This attributes credits and tokens to the specific query.

   You should see: Token and credit figures for the transcription query, once the Account Usage data has arrived.