TL;DR: Automated browser QA on every pull request means running a real browser against the PR's preview deployment — clicking through the changed flow like a user — before a human reviews it. Traditional CI (lint, unit tests, type checks) proves the code compiles; it can't prove the button works. Add a browser-QA step that reads the diff, exercises the affected flow on the preview URL, and posts a pass/fail verdict with a recording. The lowest-maintenance way to do this is an AI reviewer like screencli that installs as a GitHub App and needs zero hand-written test files.
What is automated browser QA on every pull request?
Automated browser QA on every pull request is a CI step that opens the app in a real browser, drives the user-facing flow that the PR changed, and reports whether it actually works — automatically, on every PR, before human review.
It runs against the preview deployment (the isolated URL Vercel, Cloudflare Pages, or Netlify builds for each branch), not a mocked DOM. The output is a verdict — pass, fail, or inconclusive — plus a recording or trace of what the browser did, posted back to the PR.
This is different from a nightly end-to-end suite or a manual QA pass. It's per-PR, it's tied to the diff, and it gates the merge with evidence a reviewer can watch in under two minutes.
Why lint, unit tests, and type checks aren't enough
Traditional CI validates the code's structure, not the running product. Each green check answers a narrower question than teams assume:
- Type checks confirm the interfaces line up — but a button whose click handler does nothing still type-checks.
- Unit tests confirm a function returns what it should against mocked inputs — but a success page that redirects to a 404 passes every unit test.
- The preview deploy confirms the bundle built and the URL returns a 200 — but the modal that never opens returns a 200 too.
The gap is behavioral: does the changed feature work when a human uses it? None of the standard checks open the app and try. As screencli's own framing puts it, your CI is green and the feature is still broken.
This gap has grown sharply with AI-generated code. According to Codacy's analysis of engineering teams adopting AI codegen, teams are opening dramatically more pull requests while review time climbs — the review queue, not the writing of code, is now the bottleneck. AI agents like Claude Code, Cursor, and Copilot produce syntactically correct changes fast, and syntactically correct is exactly the kind of "wrong" that unit and type checks wave through.
What browser QA catches that traditional CI misses
Real-browser checks surface the failures that only appear when the app actually renders and responds:
- Dead interactions — a button or link that renders but does nothing on click.
- Broken navigation — a flow that lands on a 404, a blank page, or the wrong route after submit.
- Silent state bugs — a cart that doesn't empty, a counter that doesn't update, a form that clears the wrong field.
- Integration breaks — a checkout that never reaches the payment form, an email confirmation that never arrives.
- Regressions in untouched flows — a change that quietly breaks a page the PR never mentioned.
These are precisely the bugs that reach production when a green CI pipeline is treated as "safe to merge."
How to add browser QA to your PR workflow
You can build this yourself or install a turnkey reviewer. Here's the shape of the workflow either way.
1. Make sure every PR has a preview deployment
Browser QA needs a live URL to test against. Connect Vercel, Cloudflare Pages, or Netlify so each PR auto-deploys to an isolated preview. According to Vercel's documentation, a preview deployment is created automatically on every push to a non-production branch, typically live within ~90 seconds — that URL is your QA target.
2. Decide what to test on each PR
Two options:
- Hand-authored specs. Write Playwright or Cypress tests for critical flows and run them in CI against the preview URL. Reliable, but every new feature needs a new test, and the suite rots as the UI changes — this is the maintenance tax that kills most in-house browser-QA efforts.
- Diff-driven, AI-planned checks. Let an agent read the PR diff, decide which user-facing flows changed, and generate the browser steps on the fly. No spec files to maintain.
3. Run the browser and capture evidence
Drive a headless Chromium session through the flow, and record it. A verdict without a recording is just another green check nobody trusts; a recording lets the reviewer see the checkout complete or the modal fail to open.
4. Post the verdict on the PR and gate the merge
Return pass / fail / inconclusive as a PR comment or status check, with the recording attached. Optionally block-merge on a fail. The goal: the reviewer opens the PR and the QA result is already there.
If you want the whole loop without maintaining any of it, screencli's GitHub App does exactly this — install it, pick the repos it watches, and every PR gets an AI-driven browser test on the preview deploy, verdict and recording posted before you review.
Build vs. buy: ways to run browser QA per PR
| Approach | What you maintain | Catches runtime UX bugs? | Adapts to new features | Setup effort |
|---|---|---|---|---|
| Hand-written Playwright/Cypress in CI | Every test + selectors + fixtures | Yes, for flows you scripted | No — needs a new test per feature | High |
| Visual regression (Percy, Chromatic) | Baselines + review of diffs | Layout/appearance only, not behavior | Partial — flags visual change, not logic | Medium |
| Manual QA before merge | Reviewer time on every PR | Yes | Yes | High (human cost) |
| screencli (AI reviewer, GitHub App) | Nothing — it reads the diff and plans checks | Yes — drives the real flow on the preview | Yes — regenerates checks from each diff | Low (install the App) |
Visual regression tools like Percy and Chromatic are worth pairing with behavioral QA — they catch pixel-level layout drift that a functional check ignores. But they verify appearance, not whether the flow completes. For "does the feature actually work," you need something that clicks through it.
Keeping it low-maintenance
The reason most teams abandon per-PR browser QA is maintenance: hand-written end-to-end suites are brittle, selectors break on every refactor, and someone has to write a new test for every feature. Three practices keep it sustainable:
- Test against the diff, not the whole app. Only exercise the flows the PR touched. Faster runs, fewer flakes, and the check stays relevant as the app grows.
- Let the plan regenerate itself. Whether AI-driven or templated, the set of checks should derive from what changed — so you're never manually syncing a test suite with the UI.
- Make failures watchable. A recording turns a red check into a two-minute review instead of a local reproduction session. That's what keeps the team trusting — and using — the gate.
An AI reviewer collapses all three into one install. It reads the diff, plans the browser checks per PR, and hands back a recording — no spec files, no selector upkeep. The same engine also runs in the agent's inner loop: npx screencli verify lets Claude Code, Cursor, or Cline check their own work before opening the PR, so fewer broken changes reach review at all.
Where screencli fits
screencli is an AI reviewer that opens your preview deployment in a real browser on every pull request, clicks through the change, and posts a pass/fail verdict — before your review starts. Install the GitHub App, pick the repos it watches, and it reads each PR's diff, plans the browser checks, runs them against the preview URL, and comments with a recording ~90 seconds after deploy. You don't write test files; the AI decides what to test based on what changed. It's built for teams shipping code written by AI agents — Claude Code, Cursor, Copilot — where the review queue is the bottleneck. One plan, $12/month, MIT-licensed to self-host. Get started at screencli.sh.
For a deeper look at the mechanics, see How to Use Playwright and Claude Code to Verify AI Agent Work and Preview Environments + Visual Verification for Agentic Coding.
FAQ
What is automated browser QA on a pull request? It's a CI step that opens the PR's preview deployment in a real browser, drives the user-facing flow that changed, and reports pass/fail with a recording — automatically, before a human reviews the PR. It verifies behavior, not just that the code compiles.
How is it different from unit tests or type checks? Unit tests and type checks validate code structure against mocked inputs. Browser QA validates the running app: it clicks the real button, submits the real form, and confirms the flow completes. It catches dead buttons, broken redirects, and silent state bugs that structural checks pass right over.
Do I need to write test scripts for every feature? Not with a diff-driven approach. Hand-written Playwright or Cypress suites require a new test per feature and ongoing selector maintenance. An AI reviewer like screencli reads each PR's diff and generates the browser checks itself, so there's nothing to maintain as the app changes.
Does browser QA run against production? No — it runs against the PR's preview deployment (the isolated URL from Vercel, Cloudflare Pages, or Netlify). That's the point: you verify the change in a live, throwaway environment before it merges to production.
Is this useful for AI-generated code specifically? Especially. AI agents produce more PRs, faster, and their output is often syntactically correct but behaviorally wrong — exactly what unit and type checks miss. Real-browser QA is the check that confirms an agent's change actually works before it lands.