Staging and Uploading Documents for Snowflake AI Document Functions

1.Why document workflows start with a stage

In most Snowflake work, you load data into tables before you query it. Document processing works differently. AI_PARSE_DOCUMENT, AI_EXTRACT and AI_COMPLETE all read a document directly from a stage. The parse-document guide says the function "processes documents stored on internal or external stages" and keeps the reading order, tables and headers. The AI_EXTRACT guide has the same design note: documents are processed straight from object storage so that data doesn't have to be moved.

These functions take a FILE object, not a path string or a VARCHAR column. The AI_PARSE_DOCUMENT reference defines its first argument, file_object, as a FILE object that points to a document on a Snowflake stage, and it refers you to TO_FILE for building one. TO_FILE takes two arguments: the stage reference and the file's relative path on that stage. So when you prepare documents, you decide which stage holds them and what path each file has on it. The extraction query then names that stage and path.

SELECT AI_PARSE_DOCUMENT (
    TO_FILE('@docs.doc_stage','research-paper-example.pdf'),
    {'mode': 'LAYOUT' , 'page_split': true}) AS research_paper_example;

The rest of this lesson follows from this design. Because the function reads the file in place, three things decide whether the call succeeds: how the stage is encrypted, how the file reached the stage, and whether the file itself is within the function's limits. The sections below cover each one in turn. How to choose a parsing mode is covered in a separate lesson.

Sources12

2.Creating a stage the functions can read

You can create a named stage in Snowsight. For an internal stage, choose Create » Stage » Snowflake Managed. For an external stage, choose Create » Stage » External Stage, then pick Amazon S3, Microsoft Azure or Google Cloud Platform. Enter the storage URL, and enable Authentication if the storage isn't public. Two settings in the internal-stage dialog matter for document work.

The first is the directory table. It is selected by default, and you can deselect it. A directory table lets you browse the files on the stage, but it needs a warehouse to refresh, so it costs credits. You can turn it on later if you skip it now. On an external stage you can also enable auto-refresh, so the directory table updates through event notifications when files are added or removed.

The second setting is encryption, and it is the one that causes the most problems later. Snowsight's guidance is: "To enable data access, use server-side encryption." If you leave the default, staged files are client-side encrypted. You can't change the setting later, because the encryption type cannot be changed after the stage is created. Fixing a wrong choice means creating a new stage and uploading the files again.

How much the setting matters depends on which function will read the files. AI_PARSE_DOCUMENT and AI_EXTRACT both accept client-side or server-side encrypted stages, including in accounts that use PrivateLink or network policies that block public access to stages. AI_COMPLETE is stricter. It lists only server-side encryption, and it doesn't currently work with custom network policies. If your workflow might later send the same documents to AI_COMPLETE, server-side encryption keeps that option open.

Sources345

3.Uploading documents: PUT, Snowsight, cloud tools and the Playground

Once a stage exists, how you upload depends on the stage type and the tool you use. For internal stages, the main method is the PUT command. You run it from the Snowflake CLI, SnowSQL or a driver. The prefix after PUT tells Snowflake which kind of internal stage you mean: @~ is your user stage, @%mytable is the stage attached to a table, and @ alone is a named stage.

 ?  file:///data/data.csv @%mytable;

Snowsight is the option without SQL, and it has more restrictions than PUT. It uploads only to named internal stages. You cannot use it to upload to user stages or table stages. The maximum file size for a Snowsight upload is 250 MB. Your role needs USAGE on the database and schema and WRITE on the stage. Only one upload runs at a time: if another upload is in progress, it has to finish before you can upload more files to the stage.

External stages don't support either method. To upload files to an external stage, use the tools from your cloud provider: Amazon S3, Microsoft Azure or Google Cloud Storage. Snowflake reads the files where they are stored. If an external stage looks empty in Snowsight, check that it has a directory table enabled and refreshed, and that the URL is correct. If the URL contains a subpath, it needs a trailing slash.

For any upload method, check the result before running any extraction. The LIST command shows what is on a user stage (LIST @~;), a table stage (LIST @%mytable;) or a named stage (LIST @my_stage;). The same check is available in the Python API:

stage_files = root.databases["<database>"].schemas["<schema>"].stages["my_stage"].list_files()
for stage_file in stage_files:
  print(stage_file)

For trying documents out before you build a workflow, the Document Processing Playground (AI & ML » AI Studio) offers a fourth way in. It has its own limits. You can add up to 10 documents, and each file can be at most 50 MB. You can upload from your local machine, which requires a personal database, or select Add from stage to pick documents from a database, schema and stage. The Playground shows AI_EXTRACT answers, the LAYOUT (Markdown) and OCR (Text) output, and generates SQL or Python snippets you can copy. If you added the files from a stage, you can open the snippets directly in Workspaces.

Sources637

4.Document requirements: formats, size, pages and resolution

A file that uploads successfully may still be rejected by the function. The upload limits above (250 MB in Snowsight, 50 MB in the Playground) only control what reaches the stage. Each function then applies its own input limits. For AI_PARSE_DOCUMENT, a file over 100 MB or a document over 2,000 pages fails with an error. A 200 MB PDF uploaded through Snowsight is therefore on the stage, but it can't be parsed.

The page_split row affects how you prepare files. The reference recommends setting page_split to TRUE to process long documents that exceed the function's token limit. However, the option works only for PDF, PPTX and DOCX files. On any other format it returns an error, so a long document in another format can't be handled this way. Accuracy has its own requirements. Text should be 8 points or larger. Decorative or script fonts may be harder to recognize, although handwriting is recognized. Page orientation is detected automatically. The function is trained on 15 languages, including English, Chinese, Hindi and Ukrainian.

AI_EXTRACT has its own list of formats. It accepts more file types than AI_PARSE_DOCUMENT, but has the same file-size limit: the reference says "The files must be less than 100 MB in size." AI_COMPLETE's limits depend on the model. All models accept .txt, .md and .pdf. Only Claude models also take Word, Excel, CSV and .xhtml files, and their per-file limit is lower.

Sources128

5.Access and compute for running functions on staged files

After the files are staged and within limits, two more conditions decide whether a query runs: permissions and the warehouse. For permissions, a user with ACCOUNTADMIN must grant the SNOWFLAKE.CORTEX_USER database role to anyone who calls AI_PARSE_DOCUMENT or AI_COMPLETE. The Playground also requires a role that has CORTEX_USER. The AI_COMPLETE guide adds that users "must also have READ access to the stage and file being processed." For stage operations, Snowsight lists these privileges: WRITE to upload, READ to view files, and OWNERSHIP to edit, clone or drop the stage.

For compute, Snowflake recommends running AI_PARSE_DOCUMENT and AI_EXTRACT queries on a warehouse no larger than MEDIUM. A larger warehouse does not make them faster. Both functions are horizontally scalable and process many documents in parallel, so more documents don't call for a bigger warehouse.

Sources158

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.Snowsight can upload a document to any internal stage, including your user stage or a table stage.Why is that wrong?

   Snowsight uploads only to named internal stages. For user stages (@~) and table stages (@%table), use PUT from the CLI, SnowSQL or a driver.

   Covered in Uploading documents: PUT, Snowsight, cloud tools and the Playground

2. 2.If Snowsight accepted a 200 MB PDF (under its 250 MB upload limit), AI_PARSE_DOCUMENT can parse it.Why is that wrong?

   The upload limit and the function's limit are separate. AI_PARSE_DOCUMENT rejects any document larger than 100 MB.

   Covered in Document requirements: formats, size, pages and resolution

3. 3.AI_PARSE_DOCUMENT can read a stage with default (client-side) encryption, so every document function can read it.Why is that wrong?

   AI_PARSE_DOCUMENT and AI_EXTRACT accept either encryption type. AI_COMPLETE requires server-side encryption, and the encryption type can't be changed after the stage is created.

   Covered in Creating a stage the functions can read

4. 4.page_split can split any long document AI_PARSE_DOCUMENT accepts, including HTML, TXT and images.Why is that wrong?

   page_split supports only PDF, PPTX and DOCX files. Other formats return an error.

   Covered in Document requirements: formats, size, pages and resolution

Practise it for real

Stage a local PDF on a server-side-encrypted named internal stage, confirm it is there, and parse it.

1. 1.In Snowsight, select Create » Stage » Snowflake Managed. Name the stage, choose server-side encryption, and select Create.

   Why: Server-side encryption lets every document function read the files, including AI_COMPLETE, and the setting can't be changed later.

   You should see: The new stage appears under Catalog » Explorer » Stages in your schema.

2. 2.From SnowSQL or the Snowflake CLI, run PUT file:///<path>/<your>.pdf @<your_stage>; with a PDF under 100 MB and 2,000 pages.

   Why: PUT uploads local files to an internal stage, and keeping the file within the limits avoids size and page-count errors.

   You should see: PUT reports the file as uploaded.

3. 3.Run LIST @<your_stage>;

   Why: Confirms the file name and path before you build the FILE object.

   You should see: Your PDF is listed with its size.

4. 4.Using a role with SNOWFLAKE.CORTEX_USER and a warehouse no larger than MEDIUM, run SELECT AI_PARSE_DOCUMENT(TO_FILE('@<your_stage>','<your>.pdf'));

   Why: TO_FILE turns the stage reference and path into the FILE object the function requires.

   You should see: A JSON result containing the document's extracted text.