Entire's Trails score pushes with review agents. Our Slop eval passed a double discount 6 of 6 times

On October 26, Entire launched a Git forge built for agents, and its headline idea replaces the pull request with a "Trail". Review agents score every push "like a jury judging an ice skating performance", and you merge once the scores stabilise. We ran the same kind of jury on code with five known bugs. Our view: the gate design is sound, but the score at the top of the page is the weakest evidence on it. The findings underneath found most of the bugs, and the scores waved them through.

What Entire built

The pull request, the post argues, "has become an audit log that only the agent reads". A Trail puts eval scores at the top instead, recalculated on every push. Each review agent "runs independently in a cloud sandbox and submits a score and a rationale". Entire's own repositories use three evals: Security, Slop and Human Review Required. Each shows "the current run alongside the last five runs", because "a single loop often misses a critical issue."

The Gates docs add the part the launch post skims. Gates are merge requirements that bring "branch freshness, human review, agent findings, and CI checks into one place". An Agent Review gate can block on unresolved findings above a chosen severity. That is a better design than the post's skating metaphor suggests.

What we ran

We did not run Entire. We ran its mechanism. The code was the invoice library from our lint-config teardown: 45 lines of Rust that pass clippy -D warnings and carry five known bugs. An order of exactly 100.00 gets no discount. VAT is charged on the pre-discount subtotal. The discount is subtracted twice. A malformed line is dropped silently. A large total saturates to u32::MAX. Four of our five behaviour tests fail on it, and between them they cover all five bugs. We removed our bug comments before review.

Each eval ran six times, matching Entire's six visible runs. The prompts were Entire's three eval names plus a fourth, Correctness ("does the code do what its doc comments say?"). Each run returned a score from 0 to 100, a verdict and findings. The model was gpt-4.1, with Correctness also run on gpt-5.1.

Results

Three things stand out.

Findings and scores disagreed. The Slop eval passed every time, at 89 or higher, while its own findings named the VAT bug in four runs. Correctness on gpt-4.1 passed one run at 90 with two real bugs listed underneath. A Trail that leads with the score leads with the number that was wrong.

The question decides what a bug is. Asked about slop, the jury saw the double discount as tidiness: "Redundant call to discount in total". Asked about correctness, the same model called it a bug. A customer charged the wrong amount does not care which eval was running.

The quiet bugs stayed quiet. The dropped line and the saturated total are the failures a test catches at once, because the output is plainly wrong. Across all twelve Correctness runs, each was named twice. The Security eval described one of them as safe.

Scores were stable, too. Slop stayed within six points across six runs. Stability is what Trails asks you to wait for, and here it was stability around the wrong answer.

What it changes

The post's instinct is right: one review pass misses things, and a diff is a poor interface for agent-written code. The users in the HN thread split predictably. One called it "a high-AI-trust approach". Another found the review agents "did a great job of catching" drift in a large monorepo, which is the kind of judgement a reader of code is good at.

What a jury of readers cannot do is run the code. Our five bugs were all visible to a careful reader, and the strongest model still named only three of them reliably. A test that computes the total for an order of exactly 100.00 needs no jury: it gets 12,000 where it expected 10,800 and fails, every time.

So use the Gates, and order them by evidence. Make the CI check gate blocking, with behaviour tests written from the requirement. Make the Agent Review gate block on findings, not on scores, and set the severity threshold low enough that "redundant call" in total still blocks. Read the scores last.

Caveats. One file, two models, our prompts rather than Entire's runners, and bugs we planted. Entire's runners may prompt differently, and its default trail-review runner assigns severities ours did not.