TL;DR: To test AI-generated code before you merge it, verify behavior, not just the diff: run the change on a preview deployment and exercise the actual user flow the PR touches, on top of your normal type checks and unit tests. Static checks confirm the code compiles; they don't confirm the feature works. The fastest way to close that gap on every pull request is an AI reviewer like screencli that opens your preview in a real browser, clicks through the change, and posts a pass/fail verdict before your review starts.
How do you test AI-generated code before merging it?
Test AI-generated code the same way a user would experience it: behaviorally, against a running build, on the exact flow the change affects. A five-step pre-merge workflow:
- Read the diff for intent, not line-by-line correctness. Ask what user-facing behavior is supposed to change.
- Run static checks — type checker, linter, unit tests. These are necessary but not sufficient.
- Deploy the branch to a preview environment so you can exercise the real, wired-up app (not a mocked component).
- Drive the actual flow the PR touches — click the new button, submit the form, load the page — and confirm it does what the PR claims.
- Capture evidence (a recording or trace) so the verdict is reviewable by anyone, not just whoever ran it locally.
The hard part is step 4. It's slow, it's manual, and with AI agents opening pull requests faster than any human can hand-test them, it's the step teams quietly skip. That's exactly where regressions slip through.
Why AI-generated code needs behavioral verification, not just static checks
AI-generated code passes static checks at a higher rate than it actually works — which makes green CI more dangerous, not less. The code looks clean, the types line up, and reviewers approve it with more confidence than they should.
The data backs this up. According to a June 2026 New Relic report, 94% of engineering leaders rate AI-generated code as higher or equal quality at the time of review — yet 82% experienced at least one production failure tied to AI-generated code in the previous six months. GitHub's own engineering blog notes that reviewers "actually feel better about approving" agent-generated code, even though it carries more hidden debt.
The surface looks fine. The behavior is where it breaks. A type checker verifies that a function's signature is correct; it cannot verify that clicking the button calls that function. That distinction is the entire problem with trusting AI output on structural signals alone — a point we cover in depth in verifying AI agent work with Playwright and Claude Code.
The failure modes unit tests and type checks miss
Unit tests and type checks catch structural problems; they miss behavioral ones — and behavioral bugs are the majority of what AI-generated code ships. One analysis found that roughly 60% of AI-code faults are "silent failures": code that compiles, passes its tests, and looks correct, but produces the wrong result at runtime.
Here are the classes of bug that sail past a green pipeline:
- The button that does nothing. The click handler is wired to the wrong function, or not wired at all. Types pass. The unit test asserts the reducer works in isolation. Nobody clicked the button.
- The success page that 404s. The redirect target is a stale route. The unit test mocked the navigation.
- The modal that never opens. A CSS or state bug hides it. The component renders fine in a test harness with no real layout.
- The form that silently drops input. Validation passes, the network call is mocked green, and the data never actually persists.
- The flow that breaks two steps downstream of the change — a place no unit test was written because no one predicted the interaction.
CodeRabbit's analysis of 470 open-source GitHub pull requests (320 AI-co-authored, 150 human-only) found AI-generated PRs averaged 1.7× more issues than human-written ones — 10.83 versus 6.45 per PR — concentrated in exactly the categories that reach production: 75% more logic and correctness errors, and 1.5–2× more security issues. Lightrun's 2026 report found that 43% of AI-generated code changes still require manual debugging in production even after passing QA and staging.
None of these are caught by tsc, a linter, or a mocked unit test. They're caught by using the app.
A workflow for catching regressions before merge
The reliable pattern is to layer verification: keep your fast static checks, then add one behavioral gate that runs the real app before merge. No single check is sufficient — each layer catches a different failure class.
| Verification layer | What it proves | What it can't catch |
|---|---|---|
Type check (tsc) |
Signatures and interfaces line up | The button does nothing on click |
| Unit tests | Isolated logic returns expected values | The success page redirects to a 404 |
| Preview deploy (200 OK) | The app built and the URL responds | The modal never opens |
| Visual regression (Percy/Chromatic) | The UI looks unchanged pixel-for-pixel | A brand-new flow that has no baseline |
| Behavioral browser test | A human could actually use the change | Nothing above — this is the backstop |
| screencli | Opens the preview, clicks through the change, returns pass/fail + recording | — |
The practical sequence for each PR:
- Let CI run the cheap, fast checks first — types, lint, units. Fail fast on structural errors.
- Deploy a preview so there's a real, running instance to test against. (If you're combining preview environments with automated visual checks, we walk through the full setup in preview environments + visual verification for agentic coding.)
- Derive the test plan from the diff. The change touched checkout? Test checkout. Don't re-run your whole suite — verify what actually changed.
- Exercise the flow end to end in a real browser and record it. Pass means a person could complete the flow; fail means they'd hit a wall.
- Gate the merge on that verdict, and keep the recording attached so review is a 30-second watch, not a re-reproduction.
This works whether the code came from Claude Code, Cursor, Copilot, or a human — but it matters most for AI-authored diffs, because those are the ones arriving in volume without a human's mental model of why the change is shaped the way it is.
Where screencli fits: browser QA on every pull request
screencli is the behavioral gate in that workflow, automated. Install the screencli GitHub App, point it at your repos, and every time a pull request opens it reads the diff, plans the browser checks that matter for that change, runs them against your preview deployment, and posts a pass/fail verdict with a video recording — usually within a couple of minutes, before your review even starts. You don't write test files; the AI decides what to test based on what changed.
It catches precisely the failure modes above: it opens the preview in a real browser, clicks the new button, and verifies the change does what it claims — the layer that type checks, unit tests, and a green deploy structurally cannot cover.
The same engine runs in your inner loop, too. Any AI coding agent that can run npx — Claude Code, Cursor, Cline, Windsurf — can call npx screencli to verify its own work before it opens the PR, so the diff that reaches you is one that already passed a real browser check.
Get started: install the screencli GitHub App and your next pull request gets an AI-driven browser test on its preview deploy — no test files to write.
Frequently asked questions
Can't I just write more unit tests for AI-generated code?
Unit tests help, but they test isolated logic against your assumptions — and AI-generated code fails most often in the integration between parts, not inside a single function. Roughly 60% of AI-code faults are "silent failures" that pass their tests and still break at runtime. You need at least one check that runs the whole, wired-up app the way a user would.
What's the difference between static analysis and behavioral verification?
Static analysis (type checking, linting, most SAST) reasons about code without running it — great for structural and security patterns, blind to runtime behavior. Behavioral verification executes the app and observes what happens: does the button work, does the page load, does the flow complete? For AI-generated PRs, you need both, because the code usually looks correct.
Do I need a preview deployment to test AI-generated code?
It's the cleanest approach. A preview deployment gives you a real, isolated, fully wired instance of the change to exercise — no mocks, no "works on my machine." Most modern platforms (Vercel, Cloudflare, Netlify) create one per pull request automatically, and tools like screencli pick up that URL and test against it.
How do I test AI PRs without slowing down the team?
Automate the behavioral gate. Hand-testing every AI-authored PR doesn't scale — teams are already seeing PR review time up 91% and some running 30 PRs a day against six reviewers. An AI reviewer that runs browser checks automatically on each PR gives you the verdict in minutes without a human in the loop for the first pass, so reviewers spend their attention on the diffs that actually failed.
Does this replace human code review?
No — it front-loads it. Automated browser verification tells you whether the change works so human reviewers can focus on whether it's the right change: architecture, edge cases, maintainability, security judgment. It removes the tedious "let me pull the branch and click around" step, not the thinking.