Structured Outputs, Tools and Agent Patterns for a Desired Output

1.Making the output machine-readable with response_format

When the next consumer of a model's output is code rather than a person, the output has to have a reliable shape. Prompt instructions are one way to ask for that. Structured outputs enforce it through a response_format field on the chat request. The field works the same way for every supported chat model, and Databricks handles the translation for each provider. Structured outputs are available on Foundation Model APIs pay-per-token and provisioned throughput endpoints. Databricks recommends them for extracting data from large numbers of documents (for example, classifying review feedback as negative, positive or neutral), for batch jobs that need a set format, and for turning unstructured data into structured data.

The schema is part of the request itself, which is why response_format is the Databricks mechanism for fixing the output shape. A step added after the LLM call is only post-processing. In a RAG chain that step applies business logic, adds citations or refines the generated text, so it is not how the shape of the response is set.

The sample below combines a system prompt that defines the model's role with a response_format (defined earlier in the same doc sample) that pins the output to a research_paper_extraction JSON schema.

messages = [{
        "role": "system",
        "content": "You are an expert at structured data extraction. You will be given unstructured text from a research paper and should convert it into the given structure."
      },
      {
        "role": "user",
        "content": "..."
      }]

response = client.chat.completions.create(
    model="databricks-gpt-oss-20b",
    messages=messages,
    response_format=response_format
)

Sources12

2.Tools: when the output is an action

Some business requests need the model to act, for example looking up an order, sending an email or querying an index, not just return text. Function calling describes the available functions to the model as JSON-schema tools. The model doesn't run them. It returns a JSON object of arguments, your code makes the call, and the result goes back to the model so it can write the final answer. Use cases include assistants that call other APIs, such as send_email(to: string, body: string), and mapping a phrase like "Who are my top customers?" to a call such as get_customers(min_revenue: int, created_before: string, limit: int). Function definitions use the same restricted JSON schema subset as structured outputs, but the key limit is lower: 16 keys instead of 64.

A retriever can be a tool too. In an agent, you don't hard-wire retrieval as a chain step. You wrap the vector index in a VectorSearchRetrieverTool from Databricks AI Bridge (databricks-langchain or databricks-openai). The model reads tool_name and tool_description to decide when to call the tool, so a vague description leads to the tool being called at the wrong times or not at all.

vs_tool = VectorSearchRetrieverTool(
  index_name="catalog.schema.my_databricks_docs_index",
  tool_name="databricks_docs_retriever",
  tool_description="Retrieves information about Databricks products from official Databricks documentation."
)

# Bind the retriever tool to your Langchain LLM of choice
llm = ChatDatabricks(endpoint="databricks-claude-sonnet-4-5")
llm_with_tools = llm. ? ([vs_tool])

Sources34

3.How much should the LLM orchestrate? Chains, agents and supervisors

Once you know the components, the last decision is who sequences them. Databricks describes a range of options and advises you to start simple, adding agentic behaviour only when you actually need it.

A single agent gets the request and any conversation history, decides whether to call tools, can loop over LLM and tool calls until it reaches its goal, and returns one combined response. Databricks calls it often the sweet spot for enterprise use cases. Because it loops, set iteration limits or timeouts. In a help-desk example, the agent answers a policy question directly but calls lookup_order(customer_id, order_id) for order status, retrying or asking the user for the ID if the tool returns "invalid order number".

In a multi-agent system, each specialized agent has its own expertise, context and tools, and a coordinator or supervisor sends each request to the right agent. That supervisor can be another LLM or a rule-based router. Databricks offers managed versions of these patterns. Knowledge Assistant builds and optimizes domain-specific chatbots through a UI. Supervisor Agent orchestrates Genie Agents, agent endpoints, Unity Catalog functions, MCP servers and custom agents; the Supervisor API docs call it the "Agent Bricks Supervisor Agent". When you need full control, custom agents authored in Python support LangGraph, LangChain, OpenAI and LlamaIndex. Custom agents aren't tied to any one pattern, so you can start with a chain and add autonomy later.

Sources567

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.Function calling is the best way to turn large volumes of unstructured text into structured records.Why is that wrong?

   For batch inference and converting unstructured data into structured data, Databricks recommends structured outputs (response_format). Function calling is for having the model trigger APIs and actions.

   Covered in Making the output machine-readable with response_format

2. 2.On a Claude endpoint you can send both tools and a response_format, so the agent calls tools and still returns schema-valid JSON.Why is that wrong?

   For Claude models, response_format can't be combined with tools or tool_choice. Split the work into separate calls.

   Covered in Tools: when the output is an action

3. 3.A multi-agent supervisor is the safest default for any production assistant.Why is that wrong?

   Databricks advises starting simple and adding agentic complexity only when needed. Chains or a single agent are cheaper to run and easier to debug.

   Covered in How much should the LLM orchestrate? Chains, agents and supervisors