
Prompts in the Pipeline: Is Your AI Reviewer Cheating?
Summary
Coding agents write code at machine speed, so pull requests are arriving faster and getting bigger. Plenty of teams have responded by putting an AI security reviewer on every PR.
It takes a dozen lines of YAML in GitHub Actions to set up, and it starts flagging real issues straight away. After a few good catches, most teams assume it’s working. In security, what a reviewer misses matters more than what it catches, so you have to measure it. We spent months doing that, and the answer depends heavily on how you test it.
A reviewer is a model plus everything around it
Under the hood, an AI reviewer has two parts. The model is the brain: which provider and version, and how hard you let it think. The harness is everything around it: the system prompt, the tools it can call (grep, file reads, symbol search, call graphs), how much code it can see beyond the diff, how it manages context, which skills it can load, and how the agent loop runs (max turns, sub-agents).
Every one of those parts is a dial, and each one moves your results. The model alone is a big one. Across the 16 models we ran on the same bugs, recall ranged from 32.5% to 77.5%, cost from $0.07 to $11.08 per PR, and review time from under 90 seconds to over 18 minutes. The prompt, the tools and the context are no different. Change one and recall, noise, cost and speed can all shift, so treat each change like a code change: it doesn’t ship until the evals say it’s better.
What to measure
Evals work like unit tests for agents. You give the reviewer a known input, compare what it finds against the expected output, and track the score over time. For a security reviewer, we’d track four things:
- Recall: how many of the real bugs it catches
- Noise: how many findings it raises for every real bug
- Cost per PR: at 1,000 PRs a month, the token bill can reach $150k a year
- Speed: time from push to comment, since most teams won’t wait more than about 10 minutes
Recall is the number that matters. If a reviewer catches a bug in one run out of five, it will miss that bug four times out of five in production.
Why public test apps give you the wrong answer
Testing a reviewer needs two things: a real codebase to explore, and exact knowledge of where every bug is.
Vulnerable-by-design apps like Juice Shop, WebGoat, DVWA, NodeGoat and the OWASP Benchmark look ideal. They aren’t, for two reasons.
First, the code gives the answers away. Many were built for pattern-matching SAST scanners and are annotated for scoring. Juice Shop’s login route tags its SQL injection with // vuln-code-snippet vuln-line on the vulnerable line. To a language model, that’s an answer key.
Second, stripping comments doesn’t help: the models have seen the code. Juice Shop has a decade of forks, an official solutions book and CTF write-ups in training data. Given a comment-stripped Juice Shop file, GPT-6.1, Claude Sonnet 5.5 and Gemini 4 Argon each named the project and the exact file.
Benchmarks also go stale once model builders train on them. When someone shows you a leaderboard, ask what the models were tested on and whether that code was public.
Building a benchmark models can’t cheat on
We tried two different approaches for building private benchmarks.
The first is synthetic. Generate a codebase loosely based on an open-source reference repo and plant a known security issue in it. It didn’t scale. Every synthetic change had to hold the planted issue and nothing else, and the code had to run as a real application so we could verify exploits against it. Checking both by hand was tedious.
The second, and far more scalable, is OSS replay. Start from the commit before a real CVE was fixed and reintroduce the bug as an ordinary-looking PR. The app already runs, the fix is the ground truth, and reviewing the fixed commit too lets you count false positives.
Here’s one replay from our set. Open WebUI’s CVE-2026-70485 is an SSRF that works through NAT64 addresses. We planted the same root cause inside a brand new feature, calendar feed subscriptions, so the reviewer had to understand how the flaw works rather than remember the CVE. The PR used no CVE wording, and the code even looked hardened.
For every bug, we kept the answer key private: the exact file and line, an exploit that works on the vulnerable version and fails on the fixed one, and a fix that never gets pushed anywhere a model could see it. Each review was graded in three independent blind passes, with a fourth pass by a different model as a cross-check.
Even then, benchmarks wear out fast. Our first set of 10 PRs with access control bugs hit 100% recall within weeks, so we rebuilt it with replayed CVEs across six languages.
What happened when we ran 16 models
We tested 16 frontier and open-weights models on 16 replayed bugs, five runs each, for 80 scored reviews per model.
The model that scored 100% recall on our first benchmark dropped to 61% on the harder one. Across all models, recall ranged from 32.5% to 77.5%, so most missed more than a third of the bugs.
Cost ranged from $0.07 to $11.08 per PR, and review time from under 90 seconds to over 18 minutes. The top scorer was far too slow to use on real PRs.
Noise ranged from about 5 to 14 findings for every real bug.
The bugs themselves split into two clear groups. Nine easy bugs were caught in 70 to 86% of reviews, while four hard ones were caught in 0 to 9%. No single model found all the hard bugs, and the SSRF went undetected in all 80 attempts.
What to take into your own pipeline
If you run an AI reviewer today, here’s where we’d start:
- Treat every part of your harness as a dial, and run your evals after every change
- Be suspicious of any score earned on a public vulnerable app
- Build your benchmark from replayed CVEs and planted bugs, keep the answer key private, and keep a few unsolved bugs in the set so it can still tell models apart
- Track noise, cost, speed and consistency alongside recall, or developers will learn to tune your reviewer out
You can build all of this with an off-the-shelf model and a CI runner. Keeping it running is the hard part. Benchmarks wear out within weeks, new models land every month, and every prompt change needs a fresh round of evals.
That’s the work Dam Secure takes off your plate. We run this testing every time we change a model or prompt in our PR reviews, and we route each type of vulnerability to the model that’s best at finding it. Learn more at damsecure.ai.





.webp)


.webp)