Cortex Search, Embeddings, CKEs and Cortex Analyst

1.Vector embeddings, tokens and context windows

Every Cortex model measures text in tokens. One token is about three quarters of an English word, or around four characters. The COUNT_TOKENS function counts the tokens in a string for a given model. This matters most for embedding models, which turn text into vectors so that semantically similar passages land close together. Each embedding model accepts a fixed number of tokens, called its context window. Cortex Search lets you pick a hosted model for its vector-search stage. These sources do not list the general Cortex AI function catalog, so this section covers embeddings as Cortex Search uses them.

If a value has more tokens than the context window allows, Cortex Search cuts it down to the window's size before embedding it. This happens both at indexing time and at serving time. Keyword retrieval is not truncated. A larger window is not automatically better, though. Snowflake recommends splitting text into chunks of no more than 512 tokens (about 385 English words), even though longer-context models exist. Smaller chunks make retrieval more precise and give a downstream LLM more relevant text. SPLIT_TEXT_RECURSIVE_CHARACTER is the built-in function for this chunking.

Sources1

2.Cortex Search: hybrid retrieval, ways to query, multi-index and access control

Cortex Search is a managed hybrid search engine over your text. Each query combines vector search for semantic similarity, keyword search for lexical matches, and a semantic reranking step. Snowflake handles embedding, infrastructure and index refreshes, which follow the same refresh behavior as dynamic tables and the service's TARGET_LAG. Its main uses are RAG for chatbots, enterprise search bars, and acting as the retrieval layer for Cortex Agents.

You can create a service with CREATE CORTEX SEARCH SERVICE or in Snowsight under AI & ML » Cortex Search. You can query it three ways: through the Python API, the REST API, or the SQL SEARCH_PREVIEW function. All three accept the same core parameters (query, columns, filter, limit, scoring options). Filters only work on columns listed in ATTRIBUTES.

SELECT PARSE_JSON(
  SNOWFLAKE.CORTEX.SEARCH_PREVIEW(
      'cortex_search_db.services.transcript_search_service',
      '{
        "query": "internet issues",
        "columns":[
            "transcript_text",
            "region"
        ],
        "filter": {"@eq": {"region": "North America"} },
        "limit":1
      }'
  )
)['results'] as results;

Multi-index queries. A service can index several columns. The SQL and Python APIs accept a multi_index_query parameter: a map from each indexed column name to text queries ({"text": ...}) or vector queries ({"vector": [...]}). Naming only the indexes you need gives more refined results and costs less, because fewer columns are searched. Multi-index services can still be searched through the REST API or without multi_index_query. In that case the search runs across every indexed column, which affects query cost.

Access control. The privileges for creating a service differ from those for querying one. Change tracking must be enabled on every underlying object. A service performs searches with owner's rights, using the same security model as other owner's-rights objects.

GRANT  ?  ON CORTEX SEARCH SERVICE transcript_search_service TO ROLE customer_support;

Sources21

3.Cortex Knowledge Extensions: sharing a search service

Cortex Knowledge Extensions (CKEs) are not a new kind of index. A CKE is a Cortex Search Service that a provider shares on the Snowflake Marketplace or through a private or organizational listing. The provider loads text into a table, creates a search service on it, and lists the service. A consumer then builds a Cortex AI application, such as a chatbot, with Cortex AI Functions or the Cortex Agent API. When a user sends a prompt, the application runs a semantic search against the CKE, and its LLM reasons over the results before answering with citations and attribution. For those citations to work, the provider should include a SOURCE_URL column in the indexed columns.

Providers get four features: content protection, management, trial support and monetization. Content protection limits the percentage of the indexed corpus a consumer can retrieve within a rolling 24-hour period. Once a consumer reaches that limit, further queries are blocked until it refreshes. Providers pay to host the service, including indexing, serving and replication. Consumers pay the provider if the CKE isn't free. CKEs are available in any region where Cortex Search is available.

Sources3

4.Cortex Analyst and semantic views

Cortex Search covers unstructured text. Cortex Analyst covers structured data. It is a fully managed text-to-SQL service, exposed as a REST API, that answers business questions in natural language. It runs the SQL it generates in your virtual warehouse under your RBAC policies. A database schema alone does not capture business definitions, so Cortex Analyst relies on semantic views: schema-level objects that describe business concepts on top of your physical tables.

Semantic views also carry descriptions, synonyms, verified examples (sample questions paired with SQL that guide query generation), access modifiers that mark facts and metrics as public or private, and custom instructions that guide Cortex Analyst's SQL generation and how it categorizes questions.

A semantic view can be authored as a YAML specification. In Semantic Studio you define the view as a YAML file and deploy it to a live Snowflake object. You can also write it as code (DDL or YAML) for version control. Legacy semantic model YAML files stored on stages still work for backward compatibility, but semantic views are the recommended approach.

Semantic View Autopilot is the AI-assisted creation wizard in Snowsight (Workspaces » Add new » Semantic View). You describe what you want, and it generates logical tables, relationships and metrics. You can seed it with example SQL, a Tableau or Power BI file, or a YAML specification. It discards invalid example queries and adds the valid ones as verified queries. It also mines the creating role's query history for relationship and verified-query suggestions.

For access, use SNOWFLAKE.CORTEX_USER, which covers all Covered AI features, or SNOWFLAKE.CORTEX_ANALYST_USER, which covers only Cortex Analyst. CORTEX_USER is granted to PUBLIC by default.

Sources456

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.Text longer than the embedding model's context window is dropped from the search entirely.Why is that wrong?

   Only the vector embedding is truncated to the context window. Keyword retrieval still uses the full text.

   Covered in Vector embeddings, tokens and context windows

2. 2.Consumers of a CKE pay to index the provider's content in their own account.Why is that wrong?

   The provider hosts the Cortex Search Service and pays for indexing, serving and replication. Consumers pay the provider only if the CKE isn't free.

   Covered in Cortex Knowledge Extensions: sharing a search service