Why your PR security reviews are so noisy (and how to fix it)
Summary
Every team that turns on AI security review goes through the same arc. Week one, it finds real issues and everyone's impressed. Week three, a one-line change gets fifteen comments, two of them contradict each other, and the SQL injection it flagged turns out to sit behind an admin-only route that casts the input to an integer anyway. By week six, developers resolve the comments without reading them.
Noise is the main way security review fails, because the real finding gets dismissed along with the fake ones. The good news is that most of it comes from a handful of fixable causes.
It also lands on reviewers who are already stretched. A Harvard working paper covering 718 companies and 49 million pull requests found that after teams adopted coding agents, review comments per PR went up 35% and the time to merge a PR went up 49%.
Every finding that isn't real takes time from a review queue that's already the bottleneck.
Where the noise comes from
Models want to find something. Ask a model whether there's a vulnerability in a diff and it'll often hunt until the answer is yes. In our September 2026 benchmark, DeepSeek V4.1 Flash found as many planted vulnerabilities as Claude Opus 5 but raised 8.8 findings for every real one, and Claude Sonnet 5 raised 14.4. Cheap per PR, expensive per developer.
The review only sees the diff. Without knowing how the rest of the codebase works, a reviewer can't tell that the URL in the webhook fetcher only ever goes to allowlisted hosts, or that order IDs get cast to integers in the gateway before they reach the query. So it flags both.
Nothing is remembered. Your team dismisses a finding and explains why. The next PR touches the same file and the same finding comes back, because PR comments don't carry state.
Severity is generic. A SQL injection on a public endpoint and one behind an audit-logged admin panel both come back as critical, so the label stops meaning anything.
What actually cuts it
The fix for the first problem is picking models by how much junk they produce, not just how much they find. Findings per real vulnerability is the number to watch. Better still, route different vulnerability types to the models that find them best and verify every finding before it gets posted.
The fix for the second is context. A review that has mapped your codebase can check each finding for reachability and existing mitigations before anyone sees it. Is the vulnerable code reachable from outside? Does something upstream already block it? Is it test-only code that never ships? Answer those and most false positives drop out on their own.
The fix for the third is memory. When someone dismisses a finding with a reason, that reason should stick, so the same false positive doesn't come back next week.
And the fix for the fourth is ranking by real risk: severity that reflects who can reach the flaw and what it would let them do. Then only block merges on the high and critical findings, and let everything else ship and get tracked.
How to tell if it's working
Track four numbers for a month. Findings per PR, and how many of those turned out to be real. How many findings get dismissed versus fixed. How long a real finding takes to get fixed.
And whether developers are reading the comments at all, which you can usually tell from how fast they resolve them. If findings per PR is in single digits and most of them get fixed, you're in good shape.
How Dam Secure keeps the noise down
AI triage on #612
Dam Secure PR Security Reviews start from a map of how your codebase fits together.
Every finding goes through AI triage that checks it for reachability and existing mitigations, confirms the ones that are exploitable in your code, and dismisses false positives automatically when confidence is high.
In practice a PR that would have raised 30 candidate findings comes back with five worth reading, each with the fix.
Every fix, dismissal and PR comment feeds into future reviews, along with your threat models and where your code runs, so a false positive your team explains once stays gone.
Teams see 3x fewer false positives, and fewer than five findings per PR instead of 15 or more.
"Our weeks-long triage has turned into days. Devs are now fixing issues live on the PR before even deploying their changes." Sebastian Vaughan, Platform Engineering Manager, Dovetail
FAQ
Why do AI security reviews produce so many false positives?
Mostly because they only see the diff, can't check whether a flaw is reachable or already mitigated, and don't remember what your team dismissed before.
How many findings per PR is normal?
For a security review developers will actually read, aim for single digits. Dam Secure averages fewer than five.
Should I tune my prompts to cut noise?
It helps a little. Codebase context, reachability checks and memory help a lot more.
Want security findings your developers will actually fix? Speak to our team.
.webp)