Farol tells a reviewer where to look. It cannot tell them whether it works

Farol is a good answer to the question a reviewer actually has when an agent opens a 40-file pull request: where should I spend my attention? Run on two real commits, it found the right blast radius, down to the function, in under two seconds, and gave the same answer twice. It also missed two kinds of change that matter on a Next.js site, and by design it never checks whether the change works. Use it to route review. Do not let a low score stand in for running the change.

Farol showed up on Hacker News today as a CLI and GitHub Action that turns a diff into a "risk map" for the approver. Its README states the premise in one line: "Agents write the code and agents review it." What is left for the human is deciding how much review a change needs.

What we ran

Farol 0.1.3 from npm, with the Next.js adapter, against two commits in the repository behind this site:

1. A content-and-template commit. It adds a post and adds a per-post cta: false flag, a two-line change to the blog loader (blog.ts) and two conditionals in the post page.
2. A navigation commit. It adds one item to the site-wide menu in Nav.tsx, plus three new routes and a robots.ts change.

Setup was one config line: { "routes": { "adapter": "next" } }.

What it got right

Function-level reach, not file-level. The changed function in blog.ts was parse. Farol listed the four routes that call it, and left out /llms.txt, which imports the same file but only uses getSlugs and a constant. A file-level tool would have flagged it, and a reviewer would have wasted a minute on it. That is the precision that makes a risk map worth reading. Not every run looked this sharp: on Farol's own backend demo, alsiola found that "the first lines suggested to look at were importing things" (comment).

The one-line nav change was the riskiest thing in either commit. Adding a menu item looks trivial in a diff. Farol ranked Nav.tsx high because twelve of eighteen pages render it, which is exactly why it deserved a look.

Deterministic. A second run of the first commit produced identical JSON apart from the timestamp and timings. That matches the README's promise: analyze "never executes code" of the repository and "never uses the network", and "the same PR gives the same report". You can diff two reports and trust the difference.

Honest defaults. With no coverage report supplied, it printed "diff coverage not verified: risk is counted as if nothing were tested", rather than assuming the tests were fine.

What it missed

Next.js metadata routes. robots.ts was reported as "no route reaches it". In the App Router, robots.ts and sitemap.ts are routes by file convention: they serve /robots.txt and /sitemap.xml. Farol's route list for the site had 21 entries and neither of those. A change that alters what search engines are allowed to crawl scored as unreachable.

Pages that live outside the import graph. The new post is a Markdown file the blog loader reads from disk at build time. Farol reported it as "new file that no route reaches", and the edit to an existing post as unreachable too. Both are public pages. Any site that renders content from files or a CMS will look smaller to Farol than it is. The README warns that "values built at runtime are invisible", and a page list read from disk is one of those. It is an expected edge for static analysis, and it is the edge most marketing and docs sites live on.

What a risk map cannot do

Farol's output ends with a "manual test checklist": /blog, /blog/:slug, /blog/authors/:slug, /feed.xml. That list is the handoff. It tells the reviewer which pages to open. It does not open them.

For the first commit, the risk that mattered was behavioural: does the post with cta: false actually render without the product card, and do the other twenty posts still render with it? The diff and the import graph are the same whether that works or not. Only running the page answers it. We checked it the boring way, by building the site and grepping the rendered HTML of both kinds of post.

The Show HN thread pulled in both directions on this. fcarraldo wanted the tool to stay small: "we need to be annotating code, not replacing the entire code review interface" (comment). timcobb pushed back on handing judgement to software at all: "If there's anything we still need a human for, it's assessing risk" (comment). Farol sits between them. It is deterministic and it never blocks a merge, so it informs the human without replacing them.

The takeaway for anyone reviewing agent-written code: a risk map shrinks the reading, which is the scarce part. Pair it with something that executes the routes it lists, or the checklist becomes the thing nobody ticks.