Documenting an AI Solution Architecture for Engineering Teams

1.What an architecture description has to pin down

Engineers who get an AI design handed to them need answers to concrete questions. Which components exist? What does each one see? What do they share? Anthropic's multiagent orchestration documentation is a good model of how to answer those questions. It begins with the purpose: one agent coordinates others to complete complex work, and the agents can act in parallel with their own isolated context. It then states exactly where the isolation boundary sits.

The key sentence divides shared resources from private ones: all agents share the same sandbox, filesystem, and vault credentials, but each agent runs in its own session thread, a context-isolated event stream with its own conversation history. Your own architecture documents need a sentence like this. An engineer who misreads the boundary will build something broken in one of two directions. They may assume a sub-agent already knows what the coordinator discussed, or they may assume files written by one agent can't be seen by another.

Behaviour over time is documented as well. Threads persist, so the coordinator can send a follow-up to an agent it called earlier, and that agent keeps everything from its previous turns. A good architecture document also says when the pattern applies. Here the doc says multiagent coordination suits complex tasks that span many surfaces, or tasks where several well-scoped pieces of work feed one goal. It names concrete patterns too, such as parallelization: fan out independent subtasks and have the coordinator synthesize the results. Write your own designs the same way. Cover the components, the boundary, behaviour over time, and the conditions under which the design is the right choice.

The same doc also records hard limits, and an architecture document should copy them, because they constrain what engineers can build. The coordinator can only delegate to one level of agents: referencing an agent that has its own roster fails the create or update request with a validation error. A maximum of 20 unique agents can be listed in the roster, although the coordinator can call multiple copies of each. Versions are pinned too. The roster is snapshotted when the coordinator is created or updated, and referenced agents stay pinned to the versions resolved at that time. They do not pick up later updates to their definitions, so the document should say that delegating to a newer version means updating the coordinator.

Claude's own guidance on project instructions offers a general pattern for the rest. It suggests describing the architecture in three sentences: what the major pieces are and how they communicate. It also lists architectural decisions specific to your project, hard constraints and known gotchas as things worth writing down. A complete architecture description therefore names the components, how they communicate, the decisions behind them, the limits engineers will hit, and the gotchas new engineers trip on.

Sources12

2.Recording why each layer exists

A component list explains what was built. Implementation guidance also has to explain why, because the team maintaining the system will eventually be tempted to add more. Anthropic's agent-building guidance gives the framing: agentic systems often trade latency and cost for better task performance, and you should consider when that trade makes sense. Workflows provide predictability and consistency for well-defined tasks. Agents fit better when flexibility and model-driven decision-making are needed at scale.

The distinction is architectural. Workflows are systems where LLMs and tools are orchestrated through predefined code paths. Agents are systems where LLMs dynamically direct their own processes and tool usage. The same guidance adds a caution that belongs in any implementation document: finding the simplest solution possible might mean not building agentic systems at all, and for many applications, optimizing single LLM calls with retrieval and in-context examples is usually enough. Start there, and record the evidence that justifies each step beyond it.

The customer support use-case guide turns this principle into an escalation path that you can copy straight into implementation guidance. As the chatbot grows more complex, the architecture can grow with it. Before adding further layers, though, the guide lists less exhaustive options: first get the most out of the prompts, then add tools (prompt chains included) and see whether they deliver the required functionality. If the bot handles incredibly varied tasks, the next step is a separate intent classifier. It sends each query through a decision tree to specialized conversations, each with its own tools and system prompt. The guide states the cost directly: the classifier needs an additional call to Claude, and that call can increase latency.

Sources34

3.From prototype to a service engineers can deploy

Implementation guidance also has to cover the step from prototype to deployable system. The customer support guide's examples are Python functions running inside a Streamlit environment. It points out that a real-time support chatbot needs an API service instead, and it gives the build steps in order. First, wrap the classification function in a simple API wrapper (Flask or FastAPI are the examples given) so that it runs as an HTTP service. Second, build a user-friendly web interface for the Claude-powered agent.

The guide also names properties the service should have, and an implementation document should turn each one into a requirement. Server-Sent Events stream responses from server to client in real time, which gives a smooth, interactive experience. Caching can shorten response times and cut unnecessary API calls. Context retention keeps a conversation continuous when a user navigates away and comes back.

Anthropic's agent guidance also warns about the tooling you choose. Frameworks simplify low-level tasks such as calling LLMs and chaining calls. However, they can add layers of abstraction that hide the underlying prompts and responses, which makes debugging harder, and they can tempt teams to add complexity that a simpler setup wouldn't need. If your guidance prescribes a framework, say what it hides and how engineers can inspect the prompts underneath.

Sources43

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.Because agents in one session share a sandbox and filesystem, they also share conversation context, so a sub-agent already knows what the coordinator discussed.Why is that wrong?

   Each agent runs in its own context-isolated session thread with its own conversation history and configuration. Only the sandbox, filesystem and vault credentials are shared.

   Covered in What an architecture description has to pin down

2. 2.A coordinator's roster can include agents that have their own rosters, and roster agents automatically pick up new versions when their definitions are updated.Why is that wrong?

   The coordinator can only delegate one level deep, and nested rosters fail with a validation error. Referenced agents stay pinned to the versions resolved when the coordinator was created or updated, and the roster holds at most 20 unique agents.

   Covered in What an architecture description has to pin down

3. 3.Adding an intent classifier in front of a chatbot is a free routing improvement and should be the first fix for varied queries.Why is that wrong?

   The guide lists prompt engineering and additional tools first. A classifier needs an extra Claude call and that call can add latency, so the guidance should record that cost.

   Covered in Recording why each layer exists

4. 4.A new AI solution should be designed as an agentic system from the start, since agents are the more capable option.Why is that wrong?

   The guidance recommends the simplest solution possible, which might mean no agentic system at all. For many applications a single optimized LLM call with retrieval and in-context examples is usually enough.

   Covered in Recording why each layer exists