What you will be able to do
- Explain how a code-review pipeline keeps false positives from reaching developers
- Pull a noisy issue category out of reporting with skip rules and per-type sensitivity while you improve its criteria
- Write severity definitions anchored in concrete categories and code examples so findings are classified consistently
1.False positives and the reviewer's credibility
A review bot is only useful if developers act on what it posts. The exam guide makes this point: if one category of finding is often wrong, developers start ignoring the accurate categories too, because they can't tell which comments to trust. The documentation in this lesson's sources doesn't put the trust argument in those words. What it does show is how much of Anthropic's own Code Review design exists to stop false positives reaching the PR.
In the managed Code Review product, several agents analyse the diff in parallel, each looking for a different class of issue. Before anything is posted, a verification step checks each result against how the code actually behaves and filters out false positives. Surviving findings are deduplicated, ranked by severity and posted inline. Each one includes an expandable reasoning section that shows why Claude flagged it and how it was verified, so a developer can check the claim instead of taking it on faith. Noise from minor findings is limited too: the example REVIEW.md caps Nits at five per review.
The docs also offer an indirect signal for checking whether developers act on findings. The analytics dashboard's Feedback metric counts review comments that were auto-resolved because someone addressed the issue.
Checkpoint 1 of 4· Put it in order
Put the stages of a managed Code Review run in order
- 1.Findings are posted as inline comments on the lines where issues were found
- 2.Findings are deduplicated and ranked by severity
- 3.Multiple agents analyse the diff and surrounding code in parallel, each for a different class of issue
- 4.A verification step checks results against actual code behaviour to filter out false positives
Verification comes before ranking and posting, so false positives are removed before a developer sees them.
“Findings are deduplicated, ranked by severity, and posted as inline comments on the specific lines where issues were found.”Source: support.claude.com
2.Pulling a noisy category while you fix it
Suppose one category, say style comments or one type of vulnerability, produces most of the false positives. The exam guide's recommended move is to disable that category temporarily. That restores trust in the rest of the review while you rewrite the criteria for the noisy category. The sources here don't describe a "temporarily disable" workflow under that name, but they document the mechanisms you'd use for it.
A skip rule. In REVIEW.md, a Do not report entry removes a category from what gets posted. Because REVIEW.md rules are added on top of the built-in correctness checks, skipping one category doesn't switch off the default bug-finding. When the improved criteria are ready, delete the entry or replace it with a precise Always check rule.
Per-type sensitivity. The automated security-review GitHub Action applies customisable filtering rules to reduce false positives. It also lets teams adjust sensitivity for individual vulnerability types, so one noisy type can be turned down without muting the others.
What these have in common is that each one acts on a single named category. A global "be more conservative" would also cut findings in the categories developers currently trust, which is the opposite of what you want.
Checkpoint 2 of 4· Check yourself
Your security-review action produces many false positives for one vulnerability type, while its other findings are accurate. Which documented control fits this situation best?
The security-review action lets you adjust sensitivity for each vulnerability type. That targets only the noisy category and leaves the accurate ones alone. A global confidence instruction affects every category, and changing the trigger only changes when reviews run.
“adjusting sensitivity levels for different vulnerability types”Source: support.claude.com
3.Severity levels defined by categories and examples
Severity labels have the same problem as "be conservative". If "critical" means "serious impact", two reviewers will rank the same bug differently. Anthropic's evaluation guide makes the same point about success criteria: after setting a goal that 90% of errors should cause inconvenience rather than egregious error, it adds that you would also need to define what "inconvenience" and "egregious" mean.
The example REVIEW.md defines its top severity by listing what qualifies for it.
## What Important means here
Reserve Important for findings that would break behavior, leak data,
or block a rollback: incorrect logic, unscoped database queries, PII
in logs or error messages, and migrations that aren't backward
compatible. Style, naming, and refactoring suggestions are Nit at
most.A definition like this is stronger when each level comes with a concrete example of a finding. The Code Review docs show what that looks like: a race condition in token refresh is Important, and a parser that silently returns 0 on bad input is a Nit.
Severity | File:Line | Issue
🔴 Important | src/auth/session.ts:142 | Token refresh races with logout, leaving stale sessions active
🟡 Nit | src/auth/session.ts:88 | parseExpiry silently returns 0 on malformed inputPutting examples into the prompt applies few-shot guidance. Anthropic calls examples one of the most reliable ways to steer output and asks that they be relevant and diverse, covering edge cases so Claude doesn't pick up unintended patterns. Wrap them in <example> tags so they stay separate from the instructions. Choose them carefully, because Claude 4.x models pay close attention to the details in examples. An example placed at the wrong level teaches the wrong boundary.
Checkpoint 3 of 4· Exam question
A reviewer flags a docstring that says "returns the cached value if present, otherwise fetches from the API," but the function under review always calls the API regardless of a cache. Under an explicit-criteria rule that only flags comments contradicted by actual code behavior, should this finding be reported?
Correct answer: A — Yes, because the docstring makes a specific, checkable claim about caching behavior that the code's actual control flow directly contradicts.
- A. Correct. The docstring asserts a specific, verifiable claim (that a cache is checked before an API call), and the code's actual control flow contradicts that claim, which is exactly the kind of concrete contradiction the explicit criterion is designed to catch.
- B. Incorrect. This treats all docstrings as aspirational rather than descriptive, which is inconsistent with the explicit criterion of flagging comments that make a specific claim contradicted by actual behavior; a docstring stating concrete behavior is exactly the kind of claim that criterion covers.
- C. Incorrect. The criterion is defined around whether the comment makes a specific claim contradicted by the code, not around whether the described behavior is an 'implementation detail'; caching behavior described in a docstring is a specific claim like any other.
- D. Incorrect. The explicit criterion is based on whether the comment's claim is contradicted by the code's actual behavior, not on how many call sites the function has; call-site count is not part of the defined rule.
Checkpoint 4 of 4· Check yourself
Which "Important" definition is most likely to make different reviewers classify the same issue the same way?
Only the third option names concrete categories and anchors them with examples, so the classification doesn't depend on each reviewer's idea of "significant". The others rely on adjectives, confidence or personal judgement.
“or block a rollback: incorrect logic, unscoped database queries, PII”Source: code.claude.com
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.A severity level is well defined by a descriptive adjective, such as "critical = serious impact on users".Why is that wrong?
An adjective still leaves each reviewer to decide what counts. Consistent classification needs severity defined by concrete categories, ideally with a code example at each level.
Covered in Severity levels defined by categories and examples
2.Adding a REVIEW.md skip rule for one noisy category also switches off Code Review's default bug-finding.Why is that wrong?
REVIEW.md rules are added on top of the default correctness checks. Skipping one category leaves the rest of the review running.
Covered in Pulling a noisy category while you fix it
Practise it for real
Give a repository's Code Review explicit severity and report-versus-skip criteria, then confirm a review runs with them
1.Create REVIEW.md at the repository root with a 'What Important means here' section that lists concrete categories, such as incorrect logic, unscoped database queries and PII in logs, and says style and naming are Nit at most.
Why: A categorical severity definition makes classification consistent, unlike an adjective.
You should see: A REVIEW.md file at the repo root. Code Review finds it automatically, with no configuration.
2.Add a 'Do not report' section covering anything CI already enforces and any generated files, and an 'Always check' section with one rule your team cares about.
Why: Explicit skip and always-check lists replace confidence-based filtering.
You should see: Two short lists, each entry naming a concrete category.
3.Open a test PR that adds a log line containing a user ID, and a lint-only formatting change. If the repo's trigger is Manual, comment @claude review at the start of a top-level PR comment.
Why: This tests whether the always-check rule fires and the skip rule holds.
You should see: A check run named Claude Code Review appears. The log line is flagged and the formatting change isn't.
Stuck? Get a nudge
If the formatting change is flagged anyway, check that the skip rule names the category explicitly. "Don't nitpick" doesn't name one.
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“a verification step checks results against actual code behavior to filter out false positives.”
↩︎ False positives and the reviewer's credibility“Feedback: Count of review comments that were auto-resolved because someone addressed the issue.”
↩︎ False positives and the reviewer's credibility“Both are additive on top of the default correctness checks.”
↩︎ Pulling a noisy category while you fix it“Both are additive on top of the default correctness checks.”
↩︎ Exam trap 2“Findings are deduplicated, ranked by severity, and posted as inline comments on the specific lines where issues were found.”
↩︎ Checkpoint - 2.https://code.claude.com/docs/en/code-reviewOfficial docs
“Report at most five Nits per review.”
↩︎ False positives and the reviewer's credibility“Reserve Important for findings that would break behavior, leak data,”
↩︎ Exam trap 1“or block a rollback: incorrect logic, unscoped database queries, PII”
↩︎ Checkpoint - 3.https://support.claude.com/en/articles/11932705-automated-security-reviews-in-claude-codeOfficial docs
“Applies customizable filtering rules to reduce false positives.”
↩︎ Pulling a noisy category while you fix it“adjusting sensitivity levels for different vulnerability types”
↩︎ Checkpoint - 4.
“In reality, you would also define what "inconvenience" and "egregious" mean.”
↩︎ Severity levels defined by categories and examples - 5.https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practicesOfficial docs
“Diverse: Cover edge cases and vary enough that Claude doesn't pick up unintended patterns.”
↩︎ Severity levels defined by categories and examples - 6.https://claude.com/blog/best-practices-for-prompt-engineeringSecondary source
“Claude 4.x and similar advanced models pay very close attention to details in examples.”
↩︎ Severity levels defined by categories and examples