We got a perfect Lighthouse score on a checkout that ships the wrong order

This week a post called "Removing Next.js Got My Blog to 100 on Every Lighthouse Category" reached 194 points on Hacker News. For a static blog, that score is most of the job, and the author earned it. But teams also put that same badge on apps with buttons. Our view: a Lighthouse 100 says the page is fast and well formed. It says nothing about whether the page did what the user asked, and for anything with a form, that is the question that pages you at 2am.

So we tried to prove it. We built a broken app, got it to 100, and then gave Lighthouse every chance to notice.

The app

A one-screen mug shop: a quantity field, a Place order button, and a status line. The click posts to /api/order and shows Order N confirmed: Q x mug. We planted one bug, the kind a coding agent leaves after a refactor: the request body sends qty: 1 no matter what the field says.

We ran a production vite build, served it with vite preview, and pointed Lighthouse 12.8.2 at it with its defaults (mobile form factor).

Run 1: navigation, the badge everyone screenshots

The first run scored 100 for performance, accessibility and best practices, and 82 for SEO. The two SEO failures were a missing meta description and a robots.txt that was really the app's HTML. We added one meta tag and a two-line robots.txt, rebuilt, and ran it again:

That is the screenshot. The bug was in the bundle the whole time. A navigation audit loads the page and stops; it never presses the button. As altmanaltman put it in the thread, "The scoring is based on how fast a page loads and what blocks its loading time."

Run 2: a user flow that actually clicks

That is not quite fair to Lighthouse, which ships user flows for exactly this. A flow drives the page with Puppeteer and audits each step. So we wrote one: load the page, start a timespan, fill the field with 3, click Place order, wait for the status line, end the timespan, then take a snapshot of the result.

The two navigation misses were zero-weight hints about unused JavaScript and the request chain. Everything that counts passed. Meanwhile, the same Puppeteer session recorded what the page did:

status text: Order 1002 confirmed: 1 x mug
requests: [{"status":200,"body":"{\"id\":1002,\"qty\":1}","sent":"{\"sku\":\"mug\",\"qty\":1}"}]

The shopper typed 3. The server was told 1, recorded 1, and the page confirmed 1. Lighthouse watched the click, measured it, checked the resulting DOM for accessibility, and passed all of it, because none of its audits asks what the number should be.

What Lighthouse verifies

The Lighthouse overview says it helps "improve the quality of web pages", with audits for "performance, accessibility, SEO, and more". It is good at that. In our flow it would have caught a layout shift during the click, a long task, an unlabelled field, or a console error. Those are real bugs, and the snapshot mode is a decent way to audit accessibility behind a button, which the docs call out.

The user-flow docs are also honest about the limits of each mode. A timespan "does not provide an overall performance score" and has no accessibility category. A snapshot "cannot analyze any issues outside the current DOM", so no network.

What it misses

Every miss here is the same miss: Lighthouse has no idea what the page is for.

• The request payload. It saw a fast POST return 200. It does not read what was sent.
• The response meaning. A 200 with qty: 1 and a 200 with qty: 3 are the same audit result.
• The rendered value. The status line is visible, labelled and contrast-compliant. That it says the wrong number is not an audit.
• Application state. There is no category for "the cart holds what the user chose".

SwellJoe's comment on the thread explains why the badge feels like more than it is: "The magic of a static HTML blog is you don't have to think about performance. An entire category of problem just disappears." On a static blog, the page is the product, so a page audit is close to a product audit. Add one form and that stops being true.

Verdict

Keep Lighthouse in CI. It is cheap, it catches regressions people do feel, and the user-flow API is underused. Just do not let a row of green circles stand in for "it works". Put one assertion next to it that knows the expected value: the request body, the response, the rendered number. We built that check for this exact bug in a separate tutorial; any test that asserts qty is 3 would do.

Caveats. One toy app, one planted bug, Lighthouse 12.8.2 with default settings on a local preview server. Lighthouse makes no claim to test business logic, so this is a test of how its score gets read, not a flaw in the tool.