Cortex Vector Embeddings, Semantic Search, RAG and Fine-tuning

1.Vector embeddings with AI_EMBED

An embedding reduces high-dimensional data such as free text to a vector of numbers. Its geometry preserves meaning: semantically similar texts produce vectors that point in roughly the same direction. Take a help desk that wants to surface already-resolved cases similar to a new one. Matching on embeddings finds related cases even when they share no keywords.

In Snowflake you create embeddings with AI_EMBED(model, input). It is the updated version of EMBED_TEXT_768 and EMBED_TEXT_1024, so treat those names as legacy. The function returns a value of the VECTOR data type. The model you pick fixes the vector's dimension, so it also fixes the column type you store it in.

Three practical points follow from the table and the AI_EMBED reference. First, models may cost different amounts, so the cheapest model that fits your language needs and text length is usually the right choice. Second, for multilingual text, choose a model marked Multilingual. voyage-multilingual-2 also has a far longer context window than the others. Third, images need a different model: for image input the only option is voyage-multimodal-3, with the image passed as a FILE via TO_FILE. Calling AI_EMBED requires a role with SNOWFLAKE.CORTEX_USER or the narrower SNOWFLAKE.CORTEX_EMBED_USER database role.

To search documents semantically, you first store their embeddings in a VECTOR column and keep them current as documents are added or edited:

ALTER TABLE issues ADD COLUMN issue_vec VECTOR(FLOAT,  ? );

UPDATE issues
  SET issue_vec = AI_EMBED('snowflake-arctic-embed-m', issue_text);

Sources12

2.Similarity functions, semantic search and RAG

Once you have vectors, you compare them with one of four vector similarity functions: VECTOR_INNER_PRODUCT, VECTOR_L1_DISTANCE, VECTOR_L2_DISTANCE and VECTOR_COSINE_SIMILARITY. Two client details come up in practice. The Python Connector supports the VECTOR type from version 3.6, and Snowpark Python from version 1.11. However, Snowpark Python has no VECTOR_COSINE_SIMILARITY function; its example uses vector_l2_distance instead.

A semantic search has two steps. Embed the search term with the same model used for the documents. Then rank the documents by similarity, using ORDER BY and LIMIT to keep the top k and, optionally, a minimum-similarity threshold.

The reason is cost. A similarity call in SELECT is evaluated only for rows that pass the other WHERE conditions, such as "logged in the last 90 days". Put it in WHERE and it can end up running across the whole table.

Retrieval-augmented generation (RAG) builds on this search. You find the documents most similar to the user's question, then pass the best match to an LLM together with the question as context for its completion. This can improve how appropriate the answer is significantly. In the example below, the wiki articles were already embedded into a vec column. The query embeds the question, retrieves the closest article, and builds the AI_COMPLETE prompt with CONCAT:

-- Embed incoming query
SET query = 'in which year was Snowflake Computing founded?';
CREATE OR REPLACE TABLE query_table (query_vec VECTOR(FLOAT, 768));
INSERT INTO query_table SELECT AI_EMBED('snowflake-arctic-embed-m', $query);

-- Do a semantic search to find the relevant wiki for the query
WITH result AS (
    SELECT
        w.content,
        $query AS query_text,
        VECTOR_COSINE_SIMILARITY(w.vec, q.query_vec) AS similarity
    FROM wiki w, query_table q
    ORDER BY similarity DESC
    LIMIT 1
)

-- Pass to large language model as context
SELECT AI_COMPLETE('mistral-7b',
    CONCAT('Answer this question: ', query_text, ' using this text: ', content)) FROM result;

Sources1

3.Cortex Fine-tuning with FINETUNE

Prompt engineering and RAG change what the model is given. Fine-tuning changes the model itself. Cortex Fine-tuning uses parameter-efficient fine-tuning (PEFT): it trains customized adaptors on top of a pre-trained model rather than training a large model from scratch. It is the option for when you need better latency and results than prompting or RAG deliver, and it runs as a fully managed service inside Snowflake. The base model documented for tuning is llama3.1-8b, with a 24k context window split into 20k for the prompt and 4k for the completion.

Everything goes through one function, SNOWFLAKE.CORTEX.FINETUNE, and its first argument selects the action:

- CREATE starts a job and returns a fine-tuned model ID. - SHOW lists all fine-tuning jobs in the account. - DESCRIBE reports a job's progress and status. - CANCEL stops a job.

You can also start a job from Snowsight under AI & ML » AI Studio. Jobs are long-running and are not tied to your worksheet session, so you check on them with SHOW or DESCRIBE.

SELECT SNOWFLAKE.CORTEX.FINETUNE(
  ' ? ',
  'my_tuned_model',
  'llama3.1-8b',
  'SELECT prompt, completion FROM my_training_data',
  'SELECT prompt, completion FROM my_validation_data'
);

Training data comes from a table or view, through a query whose result has columns named prompt and completion. If your columns have other names, alias them (SELECT a AS prompt, d AS completion ...). Any other columns are ignored. Each pair should show the exact response you want for that prompt. Start with a few hundred examples: many more can lengthen tuning drastically for little gain. Pairs longer than the context allotment are truncated, which can hurt model quality. The row limit falls as epochs rise. llama3.1-8b allows 150k rows for one epoch but 50k at the default of three, because the effective limit is the one-epoch limit divided by the number of epochs.

Privileges. The role creating the job needs USAGE on the database the data is read from and CREATE MODEL (or OWNERSHIP) on the schema the model is saved to. It also needs SNOWFLAKE.CORTEX_USER, which ACCOUNTADMIN grants. Other roles need a usage grant on the model before they can call it.

Cost. Training is billed per trained token, which equals input tokens multiplied by epochs. Inference with AI_COMPLETE on the tuned model is billed per token processed. Storage for the adaptors and warehouse time for the SQL commands are charged on top. SNOWFLAKE.ACCOUNT_USAGE.CORTEX_FINE_TUNING_USAGE_HISTORY shows credit and token consumption. Once the job finishes, you call the model by its returned name in AI_COMPLETE.

Sources3

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.Snowpark Python DataFrames can call VECTOR_COSINE_SIMILARITY just like SQL.Why is that wrong?

   Snowpark Python supports the VECTOR type but not the cosine similarity function. Its example uses vector_l2_distance; otherwise use SQL or the Python Connector.

   Covered in Similarity functions, semantic search and RAG

2. 2.Put the similarity threshold directly in WHERE so non-matching rows are skipped early.Why is that wrong?

   Compute similarity in SELECT under an alias and filter on the alias. The function then runs only on rows that pass the other WHERE conditions.

   Covered in Similarity functions, semantic search and RAG

3. 3.Load as many training rows as possible into a fine-tuning job; more data always gives a better model.Why is that wrong?

   Snowflake advises starting with a few hundred examples. Many more can lengthen tuning drastically for minimal improvement, and row limits fall as epochs rise.

   Covered in Cortex Fine-tuning with FINETUNE