Create a Vector Search (AI Search) Endpoint and Index on Databricks

1.What has to be in place first

The exam guide says "Vector Search", but the current documentation calls the product Databricks AI Search. It is the same product under a new name. The SDK package is databricks-ai-search, its client class is AISearchClient, and the REST paths still start with /api/2.0/vector-search/. You need two things before you can run a single similarity query: an endpoint to serve the index and the index itself. You can build both from the UI, the Python SDK or the REST API.

The full prerequisite list is short. You need a Unity Catalog enabled workspace, serverless compute turned on, and, for standard endpoints, a source table with a change data feed. The source can be a Delta Lake table, a streaming table, or a managed Iceberg v3+ table. The index is created as a Unity Catalog object, so you need CREATE TABLE on the target schema. Rights to create and manage endpoints are a separate matter, controlled by endpoint access control lists. In a notebook, install the SDK with %pip install databricks-ai-search, restart Python, and import AISearchClient from databricks.ai_search.client.

Sources1

2.Creating the endpoint: Standard or Storage Optimized

In the UI, go to Compute → AI Search → Create endpoint, enter a name, and choose a Type. In code it is a single call. The type can only be set when the endpoint is created, and it affects almost everything you do later with indexes on that endpoint.

client.create_endpoint(
    name="vector_search_endpoint_name",
    endpoint_type="STANDARD" # or "STORAGE_OPTIMIZED"
)

On a standard endpoint, target_qps provisions extra throughput. You pay for that capacity whether or not queries use it. Sizing and cost decisions belong to a separate objective. What matters here is that the type you pick now decides which sync mode, filter syntax and index subtype you can use later.

Sources21

3.Choosing the index type and the embedding source

The first decision is how the index is kept up to date. A Delta Sync Index follows a source table and updates incrementally as that table changes. A Direct Vector Access Index has no source table. You read and write vectors and metadata yourself through the REST API or SDK, and you cannot create one in the UI. A Delta Sync Index built without any embedding columns is a dedicated full-text index (Beta). That subtype is only available on storage-optimized endpoints with triggered sync.

For a Delta Sync Index you then decide where the embeddings come from. With Compute embeddings, you point at a text column and Databricks embeds it through a serving endpoint. For production on standard endpoints, the docs recommend databricks-qwen3-embedding-0-6b on provisioned throughput. With Use existing embeddings, you point at an array[float] column and supply its dimension. This choice is permanent: a self-managed embedding index cannot be converted to a Databricks-managed one later, so switching means building a new index.

There are a few practical details to watch. _id is a reserved column name, so rename any source column called _id first. The index name must use the three-level form <catalog>.<schema>.<name>. If you restrict Columns to index, keep in mind that only indexed columns can be returned in results or used as filters. If Databricks computes the embeddings, turn off *Scale to zero* on the embedding endpoint, because a cold endpoint can make the first query time out. CPU embedding endpoints are only suitable for small datasets and tests.

index = vsc. ? (
  endpoint_name=ai_search_endpoint_name,
  source_table_name=source_table_fullname,
  index_name=vs_index_fullname,
  pipeline_type='TRIGGERED',
  primary_key="id",
  embedding_source_column="text",
  embedding_model_endpoint_name=embedding_model_endpoint
)

Sources1

4.Sync mode, triggering a sync, and waiting for ONLINE

A Delta Sync Index uses one of two pipeline types. Continuous sync keeps the index within seconds of the source, but it costs more because a compute cluster runs a streaming pipeline the whole time. With Triggered sync, nothing runs until you start a sync through the SDK or REST API, for example index.sync(), or databricks vector-search-indexes sync-index from the CLI. On standard endpoints, both modes process only the rows that changed. On storage-optimized endpoints only Triggered is allowed, and each sync partially rebuilds the index.

Building a new index takes a while. The SDK sample polls describe() until detailed_state starts with ONLINE, and only then sends queries:

import time
while not index.describe().get('status').get('detailed_state').startswith('ONLINE'):
  print("Waiting for index to be ONLINE...")
  time.sleep(5)
print("Index is ONLINE")

Sources13

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.A storage-optimized endpoint can run a Continuous sync so the index stays fresh within seconds.Why is that wrong?

   Storage-optimized endpoints only allow Triggered sync. Continuous sync requires a standard endpoint.

   Covered in Sync mode, triggering a sync, and waiting for ONLINE

2. 2.Any index type can be created from Catalog Explorer with Create → Vector search index.Why is that wrong?

   A Direct Vector Access Index can only be created through the REST API or the SDK.

   Covered in Choosing the index type and the embedding source

3. 3.You can start with precomputed embeddings and later switch the same index to Databricks-computed embeddings.Why is that wrong?

   The conversion is not possible. You would need to create a new index and recompute the embeddings.

   Covered in Choosing the index type and the embedding source