Teardown: BeautifulRust's lint config went green on five live bugs

BeautifulRust is a set of Agent Skills that teach coding agents to write idiomatic Rust, "plus a tested clippy config" to hold the line in CI. The repo was created on October 23 and had 1,311 stars by October 28. We ran that config on a small crate with six planted bugs. Our view: a clean run under this config shows an agent wrote tidy Rust. It does not show the Rust works, and the cheapest way to satisfy its loudest lint turned a crash into a silent wrong answer.

What it claims

The README splits the work in two: "The skills teach the rules; the lints keep them from slipping back in." The lint half is a [lints] block for Cargo.toml and a clippy.toml. It switches on clippy's all and pedantic groups, then adds stricter rules: no unwrap() outside tests, no dbg! or todo!, no lossy as casts, a reason on every suppression and a # Panics section on any function that can panic. CI runs it with -D warnings, so any hit fails the build.

The project also checks its own config. scripts/check-lints.sh runs clippy on a crate with one deliberate violation per lint. We ran it on clippy 0.1.93 and got "28 expected lints fired". The config does what it says.

The README never claims that clippy finds logic bugs. The question here is narrower: when an agent's pull request comes back green under this config, what has been checked?

What we ran

We wrote a 46-line invoice library: parse SKU,QTY,UNIT_CENTS lines, take 10% off orders of 100.00 or more, then add 20% VAT on the discounted amount. We pasted in BeautifulRust's lints.toml and clippy.toml unchanged (commit 58757ee) and planted six bugs:

1. A malformed line panics (unwrap() on parse).
2. An order of exactly 100.00 gets no discount (> where >= was meant).
3. VAT is charged on the pre-discount subtotal.
4. A leftover dbg!.
5. The discount is subtracted twice.
6. The total is cast to u32 and truncates on a large order.

That is crate A, the first draft. Crate B is the same code after the smallest edits that make cargo clippy --all-targets -- -D warnings pass. Those were three edits plus one pair of doc backticks:

// A                                         // B
.map(|l| { ... p.next().unwrap() ... })      .filter_map(|l| { ... p.next()? ... })
dbg!(d);                                     // removed
(sub - d - discount(sub) + tax) as u32       u32::try_from(...).unwrap_or(u32::MAX)

Then we wrote five behaviour tests, about 30 lines, stating what the invoice must do. The tests live in the same crate under the same config, and the config's allow-unwrap-in-tests is what lets them use unwrap and assert.

Results

On crate A the config flagged three of the six bugs, and all three were in its own territory: a panic, a debug leftover and a lossy cast. The three arithmetic bugs never came near a lint. On crate B the config passed with exit code 0 while five bugs were live. Five tests caught all five.

The part that should worry you

Two of the "fixes" made the program worse in a way no lint can see. Bug 1 used to crash on a malformed line, and a crash is loud. After filter_map and ?, the bad line disappears and the invoice is wrong. Bug 6 used to truncate to 1,783,793,664 cents. Now it returns u32::MAX, a confident number that is just as wrong.

This is not something BeautifulRust's skill would tell an agent to do. Its error table maps "The caller may need to know what went wrong" to Result<T, E>, and it tells agents to "Collect fallible steps into Result<Vec<_>, E> and apply ?". An agent that loaded the skill might well have got this right. But the lint cannot tell the two apart. unwrap_used is satisfied by a correct Result and by a silent drop alike, and an agent told only to make CI green will find the cheaper one.

What it verifies, and what it misses

Verifies: panics outside tests, lossy casts, leftover debugging, undocumented unsafe, suppressions with no reason, and API-shape smells such as bool parameters. All of it is mechanical and fast, and the self-check means the table cannot silently rot.

Misses: whether the code does what it was asked to do. Boundaries, formulas, double application and silently swallowed input all compile cleanly and lint cleanly.

Caveats. One crate, bugs we chose, and B is a deliberately lazy agent. We did not run the skills with a model, only the lint config, which is the part that sits in CI.

How to use it

Adopt the config. It is well chosen and it is self-tested. Just do not let a green clippy job be the merge gate for agent-written code on its own. Put a behaviour check next to it that states the requirement in numbers, such as "exactly 100.00 gets the discount", because that is the claim a lint cannot make. Then watch for the pattern this run surfaced: when an agent fixes a lint by making a failure quiet instead of returning an error, treat the diff as a behaviour change and review it as one. It is the same gap we describe in testing AI-generated code.