What you will be able to do
- Run discovery for an AI use case and get out of it a problem brief and measurable success criteria
- Turn a user need into a design by following the Description Chain and assigning an AI collaboration mode to each step
- Hand engineering a set of pass/fail acceptance tests that serve as the shared definition of done
Key concept
Shared definition of done — Each early phase of an AI solution produces criteria concrete enough to pass or fail. Discovery produces success criteria, design produces acceptance tests and handoff produces an eval suite. Each later phase is judged against those criteria, not against what someone remembers being agreed.
1.Discovery: frame the problem before anything is built
An AI engagement starts with a problem, not a model. The Academy's builder course uses a short scenario: a community health clinic needs a way for patients to check wait times before coming in, and that's all you've been told. You write no code in this phase, and that's deliberate. Discovery means answering plain-language questions. Who are the users, specifically? Patients, but also staff and anyone else who touches the system. What outcome do they actually want, as opposed to the feature they asked for? What would a great solution feel like to use? And what are the real constraints?
The constraints question is the one teams skip most often. The course spells it out: "What are the real constraints? Think budget, technical skill at the clinic, patient access to devices, privacy requirements." From there, delegation is reframed as a decomposition exercise. You don't ask "should I use AI here?" You break the customer problem into steps and then decide what role AI plays in each one.
The second output of discovery is a set of success criteria. Anthropic's evaluation guidance says good criteria are Specific, Measurable, Achievable and Relevant. Achievable is where expectation-setting happens: targets should come from benchmarks, prior experiments or expert knowledge, and should not be unrealistic for current frontier models. Saying that during discovery is how you align stakeholders on what the AI system can't do before it becomes a complaint during the pilot. Most use cases also need several criteria at once, such as accuracy, toxicity, error severity and latency, rather than a single score.
| Property | What it demands | Example from the guide |
|---|---|---|
| Specific | Clearly define what you want to achieve | "accurate sentiment classification" instead of "good performance" |
| Measurable | Quantitative metrics or well-defined qualitative scales | F1 score of at least 0.85 on a held-out test set |
| Achievable | Targets based on industry benchmarks, prior experiments, AI research, or expert knowledge | A 5% improvement over the current baseline |
| Relevant | Aligned with the application's purpose and user needs | Citation accuracy critical for medical apps, less so for casual chatbots |
Checkpoint 1 of 6· Check yourself
A sponsor's brief says the pilot model "should classify sentiments well." Which rewrite is a usable discovery-phase success criterion?
The F1 version is specific, measurable and tied to a defined test set. "Never misclassify" fails the Achievable test, and the other two are not measurable.
“The sentiment analysis model should achieve an F1 score of at least 0.85”Source: platform.claude.com
Checkpoint 2 of 6· Exam question
A solutions team is scoping a new internal tool that must run long, asynchronous multi-step investigations across several backend systems, and the customer explicitly wants to avoid operating any agent-loop or sandbox infrastructure themselves. During the discovery phase, which approach should the team recommend?
Correct answer: B — Adopt Claude Managed Agents, since Anthropic operates the sandbox and session infrastructure needed for long-running asynchronous work
- A. Plausible for teams wanting full control, but requires the customer to build and operate orchestration infrastructure, which conflicts with the stated goal.
- B. Correct - Managed Agents runs the agent loop and sandbox on Anthropic-managed infrastructure, matching the requirement to avoid operating that infrastructure.
- C. The Agent SDK is a library that runs inside the customer's own process, so the customer would still be operating the runtime infrastructure.
- D. The Files API and prompt caching manage documents and context costs; neither provides the orchestration loop the tool needs.
2.Design: translate the need, then start simple
Design turns the discovery brief into something buildable. The builder course calls this path the Description Chain: a messy human need becomes a requirement, the requirement becomes a technical spec, and the spec becomes an instruction AI can act on. Prompt engineering is only the last link. The builder translates at every step, and the course is blunt about why that matters: "AI cannot hear what the user did not say." Each step also involves judgment calls the previous phase didn't make for you. A requirement that says "fast" has to say how fast, and "simple" has to describe what simple means for a patient checking from a phone in a parking lot.
Checkpoint 3 of 6· Put it in order
Put the links of the Description Chain in order, from where design starts to where AI acts.
- 1.User voice
- 2.Technical spec
- 3.AI instruction
- 4.Requirement
The chain runs from the human need, through a measurable requirement and a structural spec, to the instruction given to the AI. A failure at any link carries forward into everything after it.
“The Description Chain connects user voice to requirement to technical spec to AI instruction.”Source: academy.claude.com
For each capability in the build, design also decides how much latitude AI gets. The course offers three collaboration modes and asks for a simple table with one row per capability, each assigned a mode.
| Mode | Who does the work | Human role |
|---|---|---|
| Automation | AI does it | You check the result |
| Augmentation | You and AI work it together | Joint author, e.g. AI proposes a technical approach and you push back on what doesn't fit your constraints |
| Agency | AI operates with latitude | You set the boundaries it works inside |
Checkpoint 4 of 6· Match them up
Match each collaboration mode to its description.
Tap a term, then the definition that fits it.
The modes differ in how much latitude AI has. Automation gives the least and Agency the most, but even Agency stays inside boundaries the builder sets.
“Automation (AI does it, you check), Augmentation (you and AI work it together), or Agency (AI operates with latitude inside boundaries you set)”Source: academy.claude.com
How much architecture should the first design have? Anthropic's guidance in Building Effective Agents is to start small: "Start with simple prompts, optimize them with comprehensive evaluation, and add multi-step agentic systems only when simpler solutions fall short." Success, it says, is about building the right system for your needs, not the most sophisticated one. This principle runs through the whole lifecycle. Design starts at the simplest level that could meet the success criteria, and later iterations add complexity only when measurement shows it helps.
3.Handoff: tests as the artifact engineering receives
The sources don't describe a formal handoff ceremony. What they describe is the artifact that crosses from design to build, and that artifact is the test suite. The builder course asks for five to seven statements defining done, each concrete enough that a stranger could say whether the finished product passes or fails. "A patient with a basic smartphone can find the current wait time in under 30 seconds without creating an account" qualifies. "The system is easy to use" doesn't. Before build starts, these statements are rewritten so code could pass or fail them, and edge cases are added: the clinic is closed, there's no data, the wait time is zero.
Why tests rather than a prose spec? The course calls tests the most precise form of description. Anthropic's engineering post on agent evals gives the practical reason: "Two engineers reading the same initial spec could come away with different interpretations on how the AI should handle edge cases." An eval suite resolves that ambiguity. Early on, evals also force the product team to say what success means for the agent at all. So a handoff that comes with tests settles interpretation questions before they turn into implementation disagreements.
Tests are still a description, and descriptions can be wrong. The course ends its build exercise by handing the prototype to someone playing a patient with a sick kid and ten minutes, then asking two questions: did the tests pass, and is the patient satisfied? If the answers differ, the problem is upstream in the chain, not in the code.
Checkpoint 5 of 6· Check yourself
An architect is preparing acceptance criteria to hand to the engineering team. Which property matters most for each criterion?
A handoff criterion works only if it has an objective pass/fail outcome. Leaving edge cases open to interpretation is exactly the ambiguity tests are meant to remove.
“Each must be concrete enough that a stranger could tell you whether the finished product passes or fails.”Source: academy.claude.com
Checkpoint 6 of 6· Exam question
Before any prompts are written, a consulting team meets with a client to scope a document-classification assistant. Which activity most correctly belongs in the discovery phase of the engagement?
Correct answer: A — Defining the use case, measurable success criteria, and an evaluation set for later validation
- A. Correct - discovery establishes what problem is being solved and how success will be measured before any design work starts.
- B. Locking a model and prompt is a design-phase activity, and doing it before requirements are defined skips discovery entirely.
- C. Cost optimization through caching and batching is a monitoring/iteration concern that comes after the solution exists.
- D. A deployment runbook is a handoff-phase artifact, produced only after the solution has been designed and tested.
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.A success criterion like "good performance" or "classify well" is fine to start with and can be made precise after the pilot.Why is that wrong?
Criteria need to be specific and measurable from discovery onward, and achievable for current models. Otherwise the pilot has nothing objective to be judged against.
Covered in Discovery: frame the problem before anything is built
2.If the handed-off acceptance tests all pass, the solution is done.Why is that wrong?
Tests encode the intent you described. If they pass and the user is still unhappy, the description was wrong, and you go back up the chain instead of declaring success.
Covered in Handoff: tests as the artifact engineering receives
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.https://academy.claude.com/courses/ai-fluency-for-builders/delegation-the-builder-s-toolkitOfficial docs
“What are the real constraints? Think budget, technical skill at the clinic, patient access to devices, privacy requirements.”
↩︎ Discovery: frame the problem before anything is built“Delegation means decomposing the problem first, then deciding what AI handles at each step.”
↩︎ Discovery: frame the problem before anything is built“Write acceptance tests before code. They give you and AI a shared definition of done.”
↩︎ Key concept“Automation (AI does it, you check), Augmentation (you and AI work it together), or Agency (AI operates with latitude inside boundaries you set)”
↩︎ Checkpoint“Write acceptance tests before code.”
↩︎ Prediction“Each must be concrete enough that a stranger could tell you whether the finished product passes or fails.”
↩︎ Checkpoint - 2.
“Your success metrics should not be unrealistic to current frontier model capabilities.”
↩︎ Discovery: frame the problem before anything is built“Most use cases need multidimensional evaluation along several success criteria.”
↩︎ Discovery: frame the problem before anything is built“Clearly define what you want to achieve.”
↩︎ Exam trap 1“The sentiment analysis model should achieve an F1 score of at least 0.85”
↩︎ Checkpoint - 3.https://academy.claude.com/courses/ai-fluency-for-builders/description-building-great-thingsOfficial docs
“The builder is the translator at every step. AI cannot hear what the user did not say.”
↩︎ Design: translate the need, then start simple“Every step involves judgment calls the previous phase did not make for you.”
↩︎ Design: translate the need, then start simple“Tests are the most precise form of description.”
↩︎ Handoff: tests as the artifact engineering receives“Rewrite them so each one is something code could pass or fail.”
↩︎ Handoff: tests as the artifact engineering receives“A passing test with an unhappy user means you described the wrong intent.”
↩︎ Exam trap 2“The Description Chain connects user voice to requirement to technical spec to AI instruction.”
↩︎ Checkpoint - 4.https://www.anthropic.com/engineering/building-effective-agentsSecondary source
“Start with simple prompts, optimize them with comprehensive evaluation, and add multi-step agentic systems only when simpler solutions fall short.”
↩︎ Design: translate the need, then start simple - 5.
“Two engineers reading the same initial spec could come away with different interpretations on how the AI should handle edge cases.”
↩︎ Handoff: tests as the artifact engineering receives“An eval suite resolves this ambiguity.”
↩︎ Handoff: tests as the artifact engineering receives