What you will be able to do
- Generate embeddings with AI_EMBED and store them in a VECTOR column of the right dimension
- Pick the correct similarity, manipulation or aggregate vector function
- Assemble a retrieval-augmented generation query from embedding, similarity search and AI_COMPLETE
- Chunk long text with the recursive and Markdown-header splitters
- Estimate input tokens with AI_COUNT_TOKENS and know where the estimate falls short
1.Turning text into vectors with AI_EMBED
An embedding is a list of numbers that represents a piece of text, where texts with similar meanings produce vectors that sit close together. That lets you search by meaning instead of by keyword. A help desk can find past cases that resemble a new ticket even when the wording is completely different.
AI_EMBED takes a model name and an input and returns the embedding, written as AI_EMBED(model, text). The result goes in a column of the VECTOR data type, declared with an element type and a dimension such as VECTOR(FLOAT, 768). The dimension must match the model's output: 768 for the m-size models, 1024 for the larger ones. EMBED_TEXT_768 and EMBED_TEXT_1024 are the older functions that did the same job.
AI_EMBED accepts text or image input. Audio and video need a different function, AI_MULTI_EMBED, which creates multimodal embeddings from text, images, audio or video and returns one or more vectors with segment metadata. The model table below lists the text embedding models.
To call AI_EMBED, or any Cortex AI function, a role needs the USE AI FUNCTIONS account-level privilege and one of the CORTEX_USER or AI_FUNCTIONS_USER database roles.
| Model | Output dimensions | Context window | Languages |
|---|---|---|---|
| snowflake-arctic-embed-m-v1.5 | 768 | 512 | English-only |
| snowflake-arctic-embed-m | 768 | 512 | English-only |
| e5-base-v2 | 768 | 512 | English-only |
| snowflake-arctic-embed-l-v2.0 | 1024 | 512 | Multilingual |
| voyage-multilingual-2 | 1024 | 32000 | Multilingual |
| nv-embed-qa-4 | 1024 | 512 | English-only |
Pick the model according to your text. Multilingual content rules out the English-only models. Long passages favour voyage-multilingual-2, the only model with a 32000-token window; every other model stops at 512. Models can also differ in cost. Embed each document once and store the vector, then refresh it whenever the document is added or edited, so searches never compare against stale text.
ALTER TABLE issues ADD COLUMN issue_vec VECTOR(FLOAT, 768);
UPDATE issues
SET issue_vec = AI_EMBED('snowflake-arctic-embed-m', issue_text);Checkpoint 1 of 6· Fill the gap
This adds a VECTOR column and fills it with embeddings from snowflake-arctic-embed-m. Which dimension goes in the column declaration?
ALTER TABLE issues ADD COLUMN issue_vec VECTOR(FLOAT, ? );
UPDATE issues
SET issue_vec = AI_EMBED('snowflake-arctic-embed-m', issue_text);The column's dimension must match the model's output. snowflake-arctic-embed-m produces 768-dimension vectors. 1024 belongs to the larger models, and 512 and 32000 are context-window sizes, not output dimensions.
Source: docs.snowflake.comCheckpoint 2 of 6· Check yourself
You need to embed product manuals written in German, French and Japanese. Many passages run to several thousand tokens. Which AI_EMBED model fits best?
The text is multilingual, so the English-only models are ruled out. The passages are long, so you need voyage-multilingual-2: it is the only model with a 32000-token window, and every other model stops at 512.
“voyage-multilingual-2 | 1024 | 32000 | Multilingual (supported languages)”Source: docs.snowflake.com
2.Comparing, reshaping and aggregating vectors
Snowflake's vector functions fall into three groups, according to what goes in and what comes out:
- Similarity functions (VECTOR_INNER_PRODUCT, VECTOR_L1_DISTANCE, VECTOR_L2_DISTANCE and VECTOR_COSINE_SIMILARITY) take two vectors and return a number. Both vectors must share the same element type and dimension.
- Manipulation functions (VECTOR_TRUNCATE and VECTOR_NORMALIZE) take one vector and return a new one, truncated or normalized.
- Aggregate functions (VECTOR_SUM, VECTOR_MIN, VECTOR_MAX and VECTOR_AVG) work element by element across a column of vectors and return one vector per group.
How you sort the results depends on which metric you pick. Cosine similarity and inner product are higher for closer vectors, so order by them descending. L1 and L2 are distances, so the nearest vector has the smallest value and you order ascending. All vector functions trade a little floating-point precision for speed and can be off by up to 1e-4, so avoid tests that require exact equality. The Snowpark Python API doesn't support VECTOR_COSINE_SIMILARITY, although it does offer vector_l2_distance.
SELECT a, VECTOR_COSINE_SIMILARITY(a, [1,2,3]::VECTOR(FLOAT, 3)) AS similarity
FROM vectors
ORDER BY similarity DESC
LIMIT 1;Where you put the similarity call affects the cost of the query. Call the function in the SELECT list and give it an alias. If you need a minimum score, filter on that alias in WHERE. That way the function runs only on rows that survive your other filters, such as a 90-day date window, instead of on every row in the table.
AI_SIMILARITY and VECTOR_COSINE_SIMILARITY do related jobs at different levels. AI_SIMILARITY takes two raw inputs, embeds both and returns their similarity in one call, which suits a quick ad hoc comparison. VECTOR_COSINE_SIMILARITY compares vectors you have already stored, which is what a search over thousands of rows needs, because each document is embedded once rather than on every query.
The four similarity functions differ in the metric they compute: VECTOR_INNER_PRODUCT returns the inner product of two vectors, VECTOR_L1_DISTANCE their L1 distance, VECTOR_L2_DISTANCE their L2 distance and VECTOR_COSINE_SIMILARITY their cosine similarity. Inner product and cosine similarity grow as vectors get closer; the two distances shrink. The call takes the two vectors as its only arguments.
SELECT VECTOR_INNER_PRODUCT(a, b) FROM vectors;The manipulation functions reshape one vector. VECTOR_TRUNCATE returns the vector cut to a smaller dimension. VECTOR_NORMALIZE rescales a vector in the L2 vector space so that its elements fall in the range [0,1] and its magnitude is 1, which makes vectors of different lengths comparable on magnitude.
The aggregate functions collapse many vectors into one, element by element, and are used with GROUP BY like other aggregates. VECTOR_SUM adds the vectors, VECTOR_AVG averages them, and VECTOR_MIN and VECTOR_MAX keep the smallest or largest value in each position. Averaging the embeddings of every ticket in a category, for instance, gives one vector per category.
Checkpoint 3 of 6· Check yourself
A support search must return issues from the last 90 days whose cosine similarity to the search term is at least 0.7. Where should the VECTOR_COSINE_SIMILARITY call go so the query does the least work?
Calling the function in SELECT and filtering on its alias means it runs only on rows that pass the other filters, such as the date window, instead of on every row in the table. ORDER BY with LIMIT applies no minimum score. AI_SIMILARITY would re-embed the stored text on every query.
“Generally, the call to the vector similarity function should appear in the SELECT clause, not in the WHERE clause.”Source: docs.snowflake.com
3.Retrieval-augmented generation in SQL
Retrieval-augmented generation (RAG) grounds an LLM's answer in your own text. The process has three steps. First, embed the user's question with the same model used for the documents. Second, rank the documents by vector similarity to the question and keep the best matches. Third, put those matches into the AI_COMPLETE prompt next to the question. Because the model answers from the retrieved text instead of only from its training, the answer can improve considerably.
-- Do a semantic search to find the relevant wiki for the query
WITH result AS (
SELECT
w.content,
$query AS query_text,
VECTOR_COSINE_SIMILARITY(w.vec, q.query_vec) AS similarity
FROM wiki w, query_table q
ORDER BY similarity DESC
LIMIT 1
)
-- Pass to large language model as context
SELECT AI_COMPLETE('mistral-7b',
CONCAT('Answer this question: ', query_text, ' using this text: ', content)) FROM result;The retrieval step is the part you can replace. Here it is a hand-written cosine-similarity query, which leaves you managing the embeddings yourself. A managed alternative is Cortex Search, which handles indexing for you. Generation is AI_COMPLETE either way: Snowflake's general-purpose function for generating a completion from a text string or image with a model you name. Its optional response_format argument makes the answer follow a JSON schema or SQL type literal that you supply, which helps when downstream code has to parse the result.
The two vectors have different dimensions (768 and 1024). Similarity functions need two vectors with the same element type and dimension, so the comparison fails. Embed the question with the same model as the documents.
Checkpoint 4 of 6· Fill the gap
In this RAG query, the best-matching wiki article has been retrieved. Which function turns it and the question into a grounded answer?
-- Do a semantic search to find the relevant wiki for the query
WITH result AS (
SELECT
w.content,
$query AS query_text,
VECTOR_COSINE_SIMILARITY(w.vec, q.query_vec) AS similarity
FROM wiki w, query_table q
ORDER BY similarity DESC
LIMIT 1
)
-- Pass to large language model as context
SELECT ? ('mistral-7b',
CONCAT('Answer this question: ', query_text, ' using this text: ', content)) FROM result;In RAG, the generation step is AI_COMPLETE: the retrieved text goes into its prompt next to the question, so the model answers from your documents. AI_EMBED and AI_SIMILARITY belong to retrieval, and AI_SUMMARIZE_AGG summarises a column without answering a question.
Source: docs.snowflake.com4.Chunking text before you embed it
Long documents have to be split before embedding, because most embedding models stop at a 512-token context window. Smaller chunks also make retrieval return a relevant passage rather than an entire manual. Snowflake provides two splitters, and both take a chunk_size (the maximum number of characters per chunk, greater than zero) and an optional overlap (the number of characters repeated between consecutive chunks). Overlap means a sentence cut at a chunk boundary still appears whole in one of the neighbouring chunks.
Two units are in play here: chunk_size counts characters, while the model's window counts tokens, so a chunk size of 512 does not mean 512 tokens. Choose a size that leaves room, and check a sample chunk with AI_COUNT_TOKENS and the embedding model. For SPLIT_TEXT_RECURSIVE_CHARACTER, overlap must also be smaller than chunk_size; by default chunks have no overlap.
SPLIT_TEXT_RECURSIVE_CHARACTER splits text on a sequence of separators. The format argument chooses the default separators; with 'none', only separators you supply in separators are used. The splitter cuts first on the coarsest separator, starting with paragraph breaks and then line breaks. It then splits again any chunk still longer than chunk_size, until every chunk fits. It returns an array of text chunks.
SPLIT_TEXT_MARKDOWN_HEADER keeps the structure of a document. It first divides the text at the Markdown headers you list, then splits each section recursively with the plain-text separators to reach chunk_size. It returns an array of objects, each holding a chunk plus the headers it sits under. Those headers can be stored as metadata and used to filter or cite results.
SNOWFLAKE.CORTEX.SPLIT_TEXT_MARKDOWN_HEADER (
'<text_to_split>',
'<headers_to_split_on>',
<chunk_size>,
[ <overlap> ]
)headers_to_split_on maps Markdown header syntax to metadata field names. { "#": "header_1", "##": "header_2" } splits at H1 and H2 headings and stores their text in header_1 and header_2. Use the Markdown splitter when the source has meaningful headings, and the recursive splitter for plain text or other formats.
Checkpoint 5 of 6· Check yourself
A team stores Markdown runbooks of several thousand tokens each, organised under H1 and H2 headings. They embed them with snowflake-arctic-embed-m for a RAG search. What should the pipeline do before calling AI_EMBED?
snowflake-arctic-embed-m stops at a 512-token context window, so long documents have to be chunked first. The source has meaningful headings, so the Markdown splitter fits best: it keeps the headers as metadata. Overlap is there on purpose, so a sentence cut at a boundary still appears whole in one chunk. nv-embed-qa-4 also has a 512-token window, because output dimension is not the same as context window.
“snowflake-arctic-embed-m | 768 | 512 | English-only”Source: docs.snowflake.com
5.Estimating input size with AI_COUNT_TOKENS
AI_COUNT_TOKENS reports how many input tokens a call would use, without running the model. You give it the function name, a model where the function requires one (AI_COMPLETE and AI_EMBED do), the input text, and any extra inputs that add tokens, such as AI_CLASSIFY's categories. Function and model names must be lowercase and the function name must start with ai_. That means it doesn't work with the SNOWFLAKE.CORTEX functions or with fine-tuned models.
SELECT AI_COUNT_TOKENS('ai_classify',
'One day I will see the world and learn to cook my favorite dishes',
[
{'label': 'travel'},
{'label': 'cooking'},
{'label': 'reading'},
{'label': 'driving'}
]
);Every extra piece of instruction adds input tokens. Giving the same four labels descriptions and adding an overall task_description raises the count from 187 to 254, and adding one few-shot example on top brings it to 298. The function counts text only, not images, audio or video. It never counts output tokens, even though output tokens are billed too. Running it costs compute only, and it works in every region, even for models that aren't available there.
How accurate the count is depends on the model. It is exact for most models and near-exact for OpenAI. For Anthropic Claude and Google Gemini it is an estimate within 3%, except that structured output on Claude adds billed input the estimate doesn't include. To see what was actually billed for both input and output, query the usage view covered in Cortex cost monitoring.
Checkpoint 6 of 6· Fill the gap
This statement estimates the input tokens of a classification call, with its four labels included. What must the first argument be for AI_COUNT_TOKENS to accept it?
SELECT AI_COUNT_TOKENS( ? ,
'One day I will see the world and learn to cook my favorite dishes',
[
{'label': 'travel'},
{'label': 'cooking'},
{'label': 'reading'},
{'label': 'driving'}
]
);The function name must be all lowercase and must start with ai_, and it must name an AI function. A name without the ai_ prefix fails, a name qualified with the SNOWFLAKE.CORTEX namespace fails because AI_COUNT_TOKENS doesn't work with the functions in that namespace, and ai_classify_text isn't an AI function name.
Source: docs.snowflake.comSources8
Practise it for real
Build a minimal RAG query in SQL over a wiki table with a text column named content
1.Run ALTER TABLE wiki ADD COLUMN vec VECTOR(FLOAT, 768); then UPDATE wiki SET vec = AI_EMBED('snowflake-arctic-embed-m', content);
Why: Each document needs a stored embedding whose dimension matches the model, which produces 768.
You should see: Every row has a non-NULL vec of 768 floats.
2.Run SET query = 'in which year was Snowflake Computing founded?'; then create query_table (query_vec VECTOR(FLOAT, 768)) and insert AI_EMBED('snowflake-arctic-embed-m', $query).
Why: The question has to be embedded with the same model as the documents, or the vectors can't be compared.
You should see: query_table holds a single row with a 768-dimension vector.
3.Inside a CTE, select w.content and VECTOR_COSINE_SIMILARITY(w.vec, q.query_vec) AS similarity from wiki w, query_table q, ordered by similarity DESC with LIMIT 1.
Why: This is the retrieval step: higher cosine similarity means closer meaning.
You should see: One row containing the article most likely to answer the question.
4.Call AI_COMPLETE('mistral-7b', CONCAT('Answer this question: ', query_text, ' using this text: ', content)) on the CTE's result.
Why: This is the generation step: the retrieved text becomes context for the answer.
You should see: A short answer based on the article's text.
Stuck? Get a nudge
If the similarity scores all look low, check that the documents and the question were embedded with the same model.
Sources
(8)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“To create a vector embedding from a piece of text, use the AI_EMBED function.”
↩︎ Turning text into vectors with AI_EMBED“it goes beyond keyword matching to semantic similarity, so related records can be found even if they don’t contain exactly the same words.”
↩︎ Turning text into vectors with AI_EMBED“Snowflake Cortex offers the EMBED_TEXT_768 and EMBED_TEXT_1024 functions and several Vector functions to compare them for various applications.”
↩︎ Turning text into vectors with AI_EMBED“In retrieval-augmented generation (RAG), a user’s query is used to find similar documents using vector similarity.”
↩︎ Retrieval-augmented generation in SQL“This can improve the appropriateness of the response significantly.”
↩︎ Retrieval-augmented generation in SQL“Generally, the call to the vector similarity function should appear in the SELECT clause, not in the WHERE clause.”
↩︎ Exam trap 2“voyage-multilingual-2 | 1024 | 32000 | Multilingual (supported languages)”
↩︎ Checkpoint“Generally, the call to the vector similarity function should appear in the SELECT clause, not in the WHERE clause.”
↩︎ Checkpoint“snowflake-arctic-embed-m | 768 | 512 | English-only”
↩︎ Checkpoint - 2.
“AI_EMBED: Generates an embedding vector for a text or image input”
↩︎ Turning text into vectors with AI_EMBED“AI_MULTI_EMBED: Creates multimodal embeddings from text, images, audio, or video”
↩︎ Turning text into vectors with AI_EMBED“your role needs the USE AI FUNCTIONS account-level privilege and one of the CORTEX_USER or AI_FUNCTIONS_USER database roles”
↩︎ Turning text into vectors with AI_EMBED“AI_SIMILARITY: Calculates the embedding similarity between two inputs.”
↩︎ Comparing, reshaping and aggregating vectors“Generates a completion for a given text string or image using a selected LLM.”
↩︎ Retrieval-augmented generation in SQL - 3.
“Similarity functions operate on two VECTOR arguments of equal element type and dimension, computing the specified metric.”
↩︎ Comparing, reshaping and aggregating vectors“Vector manipulation functions take an existing vector and return a new vector with different properties, such as truncation or normalization.”
↩︎ Comparing, reshaping and aggregating vectors“These functions have a margin of error up to 1e-4.”
↩︎ Comparing, reshaping and aggregating vectors - 4.
“Normalizes a VECTOR in the L2 vector space, giving its elements values in the range of [0,1] and giving it a magnitude of 1.”
↩︎ Comparing, reshaping and aggregating vectors“Truncates a VECTOR to a smaller dimension.”
↩︎ Comparing, reshaping and aggregating vectors“Computes the element-wise average of vectors in an aggregate.”
↩︎ Comparing, reshaping and aggregating vectors - 5.
“AI_COMPLETE lets you supply a JSON schema or SQL type literal that completion responses must follow”
↩︎ Retrieval-augmented generation in SQL - 6.https://docs.snowflake.com/en/sql-reference/functions/split_text_recursive_character-snowflake-cortexOfficial docs
“Splitting is then applied to each chunk that is longer than the specified chunk_size, recursively”
↩︎ Chunking text before you embed it“If overlap is specified, it must be smaller than the chunk_size argument.”
↩︎ Chunking text before you embed it“An integer specifying the maximum number of characters in each chunk.”
↩︎ Chunking text before you embed it - 7.https://docs.snowflake.com/en/sql-reference/functions/split_text_markdown_header-snowflake-cortexOfficial docs
“The function returns an array of objects, where each object contains the text chunk and the associated headers under which that chunk falls.”
↩︎ Chunking text before you embed it“An integer specifying the number of characters to overlap between consecutive”
↩︎ Chunking text before you embed it - 8.
“AI_COUNT_TOKENS does not work with LLM functions in the SNOWFLAKE.CORTEX namespace or with fine-tuned models.”
↩︎ Estimating input size with AI_COUNT_TOKENS“AI_COUNT_TOKENS accepts only text, not image, audio, or video inputs.”
↩︎ Estimating input size with AI_COUNT_TOKENS“AI_COUNT_TOKENS only incurs compute costs and does not bill based on token count.”
↩︎ Estimating input size with AI_COUNT_TOKENS“The following example adds per-label descriptions and an overall task description to the previous example:”
↩︎ Estimating input size with AI_COUNT_TOKENS“This value does not include output (generated) tokens.”
↩︎ Exam trap 1“the billed input token count for these requests can be materially higher than the estimated token count.”
↩︎ Prediction