PHP's CVE week on HN: finding bugs got cheap, proving them did not

PHP shipped fixes for two remote code execution bugs on October 21, and the Hacker News thread on the PHP Foundation's write-up ran to about 400 comments. Much of it argued about who broke the embargo. The more useful ones were about the work on either side of the leak, and they point one way. AI tools made finding a bug cheap. They did not make it cheaper to prove a report is real, choose the fix, or build and ship it in private. Our view: for anyone running a project in 2026, the bottleneck has moved from discovery to verification, and the release pipeline is part of verification.

The numbers the PHP team published

A week before the release, Alexandre Daubois of the PHP security team described the queue. Since July 1 the team published 15 CVEs and closed 106 reports that "did not constitute vulnerabilities, were duplicating something already in the queue, or didn't reproduce."

That is 121 reports with a result, of which 15 were real: about one in eight. Each of the other 106 still needed a person to read it and try to reproduce it, because "no reporter hears back about an analysis a person hasn't attempted to reproduce." The post names the cost exactly: "Raw model output with a crash that doesn't reproduce takes us just as long to rule out as a real bug takes to confirm."

The team uses AI on its side too. OpenAI's Daybreak Red follows a proof of concept "through the engine" and checks "whether the claimed impact holds up". It also groups reports that share a root cause. The reproduction builds run AddressSanitizer and UndefinedBehaviorSanitizer and are "set up like our CI", because "a crash that only shows up on one custom build doesn't tell us much about the PHP you deploy."

What the threads said

We read the PHP release thread and the Patch the Planet thread from the same day. Three points came up again and again.

A claim is not a reproduction. The night before the release, someone tweeted that they had "recreated & lab/field tested" both CVEs. danso pointed out that the same author said a couple of hours later that the PoC had not been proven to work "in the wild". andrewaylett described scanning tools as "prone to flagging false positives as 'vulnerabilities' rather than 'potential vulnerabilities'". sumtechguy had the same problem at a smaller scale: an AI asked for a test case for a specific bug "kept telling me that it had it. When I could clearly trace thru the code and see it did not." After 5 to 10 rounds, they wrote the test themselves.

The fix has to be built somewhere private. emilburzo laid out the problem for an open-source project. The source is public, so you can't push the fix. The build pipeline is public, so you need "a private build system". Private builds get fewer resources, so it gets simplified, and "errors because you exercise it rarely" land on release day. modeless replied: "You should always have a local build process that replicates CI." Leherenn explained why that is harder than it sounds. Their release artifacts are signed with a key that only allowlisted build nodes can reach, and "I could not create a release build locally in a few minutes."

Shipping means the channel users actually update from. dofm noted that "for all the long-tail users, PHP is a distro package," and that route got the most work. The Foundation's post says the same. It moved the release up one day, not to that evening, because "distributors were still packaging the fix."

Where the time went

Here is the release in stages, with what made each one slow:

Only the first row got clearly cheaper, and it is the row that feeds all the others. More candidates mean more reproductions, more ruled-out reports, and more emergency builds.

The takeaway

pseudony said it in another thread about AI rewriting legacy code. It "works really well when you have an oracle (something that you can test against and that is comprehensive)". The PHP team's reproduction environment is that oracle for security reports: sanitizer builds that match CI, on every platform PHP supports. It is what turns a confident report into a yes or a no.

Two things are worth copying. First, make "it reproduces on a build like ours" the bar before anyone works on a report, whether a person or a model wrote it. Second, run your private release build as a drill, on a quiet day, before you need it. If you only build it during an embargo, release day is its first real test.