TL;DR: AI code reviewers read your diff — and the better ones read your whole repo — but none of them run your app. So they miss the bugs that only exist at runtime: the button that renders but doesn't fire, the flow that dead-ends after the change, the regression two screens away. In independent 2026 testing, leading AI reviewers caught real runtime bugs with just 42–48% accuracy in single-pass mode. The fix isn't more context or a smarter model — it's a different layer entirely: verification that runs the change in a browser before you review it. That's what screencli does on every pull request.
Two kinds of AI reviewer, one shared blind spot
AI code review comes in two flavors, and both are worth running.
Diff-only reviewers — CodeRabbit is the archetype — read the changed lines plus limited surrounding context. They're fast, low-noise, and good at local logic errors: a swallowed exception, a missing null check, an inconsistent name. According to an independent Panto.ai comparison, CodeRabbit averages about 2 false positives per PR, which is low enough that engineers don't build a "skip this bot" habit.
Codebase-aware reviewers — Greptile, Sourcegraph Cody — index the entire repository into a graph and reason about a change in that wider context. That lets them catch cross-file bugs a diff-only tool can't: "this function is called from six places and you only updated four." On its own benchmark, Greptile caught around 82% of bugs — well ahead of diff-only tools — but the same wide context produces noise, with roughly 11 false positives per PR in that Panto.ai test.
Here's the part the tool comparisons under-sell: both flavors read code without running it. More context makes the reading better. It does nothing about the fact that neither one ever runs the change. As one 2026 analysis put it, static review "can reason about code but doesn't observe what a running application does to a user's task."
The bugs that only exist at runtime
A whole class of bug is invisible to any tool that inspects source instead of executing it. The code looks reasonable in the diff — often especially reasonable, because AI writes idiomatic, confident-looking code — and breaks the moment a user touches it.
The recurring offenders:
- The dead button. The handler is wired, the props type-check, the diff is clean — and the click does nothing because an upstream state never initializes.
- The broken flow. Each step is correct in isolation, but the change reorders an effect and the checkout now dead-ends on step three.
- The off-screen regression. The PR touches the profile page; the bug lands on the dashboard, which shares a component the diff never mentions.
- The state- and timing-dependent failure. Race conditions, auth/session flows, and live-integration paths depend on runtime conditions no amount of reading can observe.
These aren't exotic. As one 2026 write-up on static analysis's limits noted, "auth, data states, race conditions, and live integrations sit in that blind spot" — and adding runtime evidence caught over 30% more issues than reading the change alone. Security researchers make the same point about AI-generated code specifically: it "often looks correct but may behave unpredictably in context," which is why runtime validation, not just static scanning, is becoming a standard layer (Invicti).
And the volume makes it worse. Teams with heavy AI adoption merge 98% more pull requests while review time climbs 91% (Faros AI, 10,000+ developers) — so more of these runtime bugs arrive, faster, than any human can manually click through. (See the AI code review bottleneck for the full data.)
Why "codebase-aware" doesn't close the gap
It's tempting to think the fix is just more context — index the whole repo and the runtime bugs fall out. They don't.
Full-repo context solves a real but different problem: cross-file reasoning. Greptile's semantic code graph catches the change that breaks a caller three files away. That's genuine coverage a diff-only tool lacks. But a graph of functions and call sites is still a static model of what the code says. It cannot tell you whether the page loads, whether the new button fires, or whether the flow completes — because those are properties of the code running, not properties of the code.
The numbers show the ceiling. Even the best static AI reviewers caught real-world runtime bugs with only 42–48% accuracy in single-pass mode, per a 2026 multi-model analysis — and more than half of their flagged issues weren't real problems. You can push recall up with more context, but you buy it with false positives, and you still never see the running app.
| Review layer | What it sees | Catches | Structural blind spot |
|---|---|---|---|
| Diff-only AI (CodeRabbit) | Changed lines + local context | Local logic errors, low noise (~2 FP/PR) | Anything outside the diff; all runtime behavior |
| Codebase-aware AI (Greptile, Cody) | Full-repo graph of code & callers | Cross-file bugs (~82% on-benchmark, ~11 FP/PR) | Still doesn't run the app — misses runtime failures |
| Deterministic gates (lint, SAST, tests) | Source + the paths tests exercise | Style, known vuln patterns, tested paths | Untested flows, UI behavior, real integrations |
| Behavioral verification (screencli) | The app running on a preview deploy | Dead buttons, broken flows, live regressions | N/A for logic-level static analysis — pair it with the above |
The layers are complementary, not competitive. Static reviewers cheaply catch what's visible in code. The gap is everything that's only visible when the code runs.
What actually catches runtime bugs: review that runs the app
To catch a bug that only appears at runtime, something has to reach runtime.
screencli is browser testing that runs on every pull request — it opens your preview deploy, drives a real browser through the flow your change touches, and reports whether the app actually works before a human opens the diff. Not "this function looks risky," but "the new button doesn't fire" or "checkout dead-ends on step three," with the click-through as evidence.
That's the layer diff-only and codebase-aware reviewers structurally can't be. It doesn't compete with CodeRabbit or Greptile — it sits underneath them, verifying behavior while they analyze code. The result is that the dead button gets caught by a machine, on the PR, instead of by your user in production or by you after you've already approved a clean-looking diff.
See screencli click through your next PR before you review it — screencli.sh.
How to layer your review pipeline
You don't pick one reviewer. You stack layers so each catches what the others can't:
- Deterministic gates first — lint, type checks, SAST, and tests run before anyone (human or AI) opens the diff. Cheapest signal, catch it early.
- Static AI review for code-level judgment — a diff-only reviewer for low-noise local bugs, or a codebase-aware one if your bugs live in the seams between files and you can absorb the false positives.
- Behavioral verification for what runs — automated browser checks on the preview deploy so runtime failures surface on the PR, not in production. (More on this in automated browser QA on every pull request.)
- Human attention for judgment — architecture, product fit, and the calls no tool should make — spent on a change that's already been read and run.
The mistake is treating any single static reviewer as "the" review. However good it is at reading code, it never runs your app — and that's exactly where AI-generated bugs like to hide.
FAQ
Why do AI code reviewers miss bugs? Because they read code without running it. Both diff-only and codebase-aware reviewers analyze source statically, so runtime failures — a dead button, a broken flow, a race condition — look fine in the diff and only appear when the app runs.
Is a codebase-aware reviewer enough on its own? No. Full-repo context fixes cross-file reasoning, not the runtime blind spot. Greptile caught ~82% of bugs on its own benchmark but logged ~11 false positives per PR, and still never executes the change.
Can't I just rely on tests to catch runtime bugs? Tests catch the paths you thought to test. AI-generated PRs frequently ship self-confirming tests that only exercise the paths the model imagined, leaving untested flows — and UI behavior in particular — uncovered.
How is behavioral verification different from AI code review? AI code review produces opinions about code it read. Behavioral verification produces pass/fail on the app it ran. The first tells you the diff looks reasonable; the second tells you whether a user can still complete the task after the change ships.
Where does screencli fit alongside CodeRabbit or Greptile? Underneath them. Keep your static reviewer for code-level feedback; add screencli to run the change in a browser on every PR so runtime bugs get caught before review — screencli.sh.