Verification vs validation, and why AI broke the distinction

Verification asks whether you built the thing right. Validation asks whether you built the right thing. That one-line answer to validation vs verification is correct, and for most of software history it was enough, because two different people answered the two questions.

That assumption no longer holds. When a coding agent writes the change, the same actor generates the code, claims it works and assesses its own claim. The classical split still describes two real questions, but it quietly assumed a third thing was already true: that the author's report could be trusted. This piece explains the distinction properly, then shows the question it now needs for agent-written code, which is the step we call AI verification.

What is the difference between verification and validation?

The short version: verification checks the work against what was specified, and validation checks the work against what was actually needed. A product can pass one and fail the other in either direction.

The table makes the difference concrete. It also includes the three activities people most often file under one of the two words, because that is where the confusion starts.

Notice the last column. Every row has a blind spot, and the blind spots do not overlap. That is the real reason the two words exist: no single activity answers both questions.

Where does the verification and validation split come from?

The distinction is older than most of the tools that now claim it. The most quoted framing is usually credited to Barry Boehm: verification is building the product right, validation is building the right product. It became popular because it fits on a slide and survives being repeated.

The standards formalise the same idea. IEEE 1012 is the IEEE standard dedicated to verification and validation of systems, software and hardware, and it treats them as two related but separate sets of activities across a project's life. ISO/IEC/IEEE 12207:2017, the international standard for software life cycle processes, lists verification and validation as distinct processes rather than one combined step.

If you want the long version, the Wikipedia entry on software verification and validation collects the definitions and the standards that use them. You do not need to read any of them to use the distinction.

What matters is the shape they agree on. Verification compares an artifact to the requirements it was built from, so it can happen early and repeatedly, at every level. Validation compares the finished thing to the real-world need, so it depends on the people who have that need and tends to happen later.

Both standards also assume something they never needed to state: the person building the artifact and the person checking it are not the same mind. Verification is independent of the author by design, even when it is done by colleagues on the same team. Hold on to that assumption. It is the one that broke.

Validation vs verification: how teams collapse the two

In practice, most teams do not get the distinction wrong on purpose. They let the words drift until both mean "we checked it". Three collapses are common enough to name.

Calling every check validation. Input validation, form validation, schema validation: the word is used so often for "the data has the right shape" that it slides into meaning any check at all. A team says the feature is validated when what they mean is that the tests passed. That is a verification claim wearing validation's name, and it hides the fact that nobody asked a user anything.

Treating UAT as verification. User acceptance testing is validation: real users, real tasks, real judgement about whether the product is acceptable. It is not designed to find a request that fires twice or a store that disagrees with the screen. When UAT is the last gate, teams assume it caught behavioural bugs it was never shaped to see.

Using "verify" to mean "I looked at it". This is the most dangerous one. "I verified it" often means someone opened the page and it looked right. Looking is evidence about the surface, not about behaviour. Some release checklists even carry a single line, verification validation, ticked by one person in one pass, as if the two were one task.

Each collapse leaves the same hole. Somebody believes a question was answered when only the other one was, or neither.

A worked example: one checkout, both questions

Take one flow and ask both questions of it. A customer adds an item to the cart, enters a card and places the order.

The validation question: is this the right checkout? The product owner wanted guest checkout with a single page, no account required, and a clear confirmation.

You put it in front of five customers. All five place an order without help, all five see the confirmation, and two say it is faster than the old one. Validation passes. This is the thing users needed.

The verification question: does this checkout do what the specification says? The spec says one order, one charge, one confirmation email, and an order record that matches the cart. Now read what the program did, not what the screen showed:

• The confirmation rendered once. Fine.
• The order record exists and matches the cart. Fine.
• The network log shows two charge requests for one click, both returning success.

The customer was charged twice. Checkout, meet double-checkout.

Verification fails, and nothing about the validation session could have found it. The customers saw exactly what they expected. The screen was correct. The bug lived in a place the validation question never looks: what the program sent over the wire.

The reverse failure is just as real. A checkout can charge exactly once, write a perfect record and still fail validation because customers abandon it at the address form. Verify vs validate is not a ranking of which matters more. They catch different failures, and a release needs both answers.

What changed when agents started authoring the change?

Go back to the assumption the standards never stated: the author and the checker are different minds. For a human team that was nearly automatic. Even a developer testing their own code tends to run the app, click through it and see something they did not predict.

A coding agent removes that separation. The same actor reads the ticket, writes the code, writes the tests, runs them and writes the summary that says the work is done. When it reports "implemented and verified", that sentence is a description of what the agent intended to build. It is not an observation of what the program does.

That is the self-assessment problem. The agent's tests encode the agent's understanding, so when the understanding is wrong, the tests are wrong in the same direction and they pass.

Its summary comes from the same understanding, and its review of its own diff reads code it already believes is correct. Every check it can run on its own work shares the blind spot of the work. The practical version of this, with Claude Code as the example, is in using Claude Code for QA automation.

So agent-written code needs a third question, asked before the other two can mean anything: did the change actually do what its author claims it did? It is narrower than verification against a spec and very different from validation with users. It asks whether the report matches reality. Until something independent of the author answers it, "verified" in an agent's summary is a claim, not a result.

Validate vs verify: three questions that tell you which one you need

When you are about to say a change is checked, ask these in order. Each one points at a different activity, and the first one you cannot answer is the work you still have to do.

1. Is the evidence about the program, or about the author's account of it? If all you have is the agent's summary, its own tests or a screenshot, you do not yet have evidence about behaviour. Get an independent check that reads what the running app did. This is the agent-era question, and it comes first because the other two depend on it.

2. Does the behaviour match what was specified? Compare what the app did, including requests, stored state and errors, against the acceptance criteria. If the spec says one charge and the network shows two, stop here. This is verification.

3. Does the specified behaviour solve the user's problem? Put the working product in front of the people who asked for it, or watch it in real use. If they cannot complete the task, the spec was wrong, however well it was built. This is validation.

Run them on Monday's merge queue and most items stop at question one, which is useful to know on its own.

Why "the tests pass" answers neither

A green test suite feels like an answer to both questions. Mechanically, it answers a third, much smaller one: do the assertions someone wrote still hold?

A test asserts on outputs its author chose to check, usually what reached the screen or a function's return value. The program, while the test ran, did more: it sent requests, changed state, logged errors, maybe fired the same request twice.

None of that is in the assertion, so none of it can fail the test. That is why a passing suite is not verification, and it is the argument made in more detail in why a green check isn't proof.

It is not validation either. A test cannot tell you whether a user wanted this feature, understood it or could finish the task. It only knows the expectations written into it.

With an agent writing both the code and the tests, the gap closes in the wrong direction. The suite becomes a description of the agent's intent, checked against itself.

Where a verification harness fits

The first of the three questions needs something that is not the author: a harness that runs the app and reads what it did. That is how the verification question gets answered mechanically instead of by assertion. What a test harness is, and why most of them read only the screen, has its own guide.

Reticle is that kind of harness. It runs inside your app in development, drives the flow your agent changed, and reads the network calls, the application state and the console, then returns a pass or fail with the evidence. The double charge in the checkout above shows up as two charge requests for one click, whatever the screen says.

It answers the agent-era question and most of verification. It does not answer validation, which still needs your users. The definition and its limits are in the verification step agents skip.

Keep reading

Start with what AI verification actually means.

Validation needs people using the product; for the human side of finding what nobody specified, read exploratory testing.

If you lead a QA team and want the same question applied to the tools you are being sold, AI QA separates what the label covers and where a model should not be the one deciding.

Fun fact

Ariane 5's first flight broke up about 37 seconds after main engine ignition, and the fault was in code that, once the launcher lifts off, "serves no purpose". It was carried over from Ariane 4, and the partners had agreed not to put Ariane 5's trajectory data into that component's requirements (Inquiry Board report, ESA summary). The software met its spec. Nobody had checked the spec against the rocket it was flying on.

Conclusion

Verification asks whether you built it right, validation asks whether you built the right thing, and a passing test suite answers neither on its own. With an agent writing the code, the tests and the summary, there is a question that comes before both: did the change actually do what its author says it did?

Next time a change lands with "implemented and verified" in the summary, run the three questions in order. If you cannot answer the first one with evidence from the running app, that is the work still left to do.

FAQ

What is the difference between verification and validation?

Verification checks that a product was built correctly against its specification: did we build it right. Validation checks that the product meets the real need of its users: did we build the right thing. Verification can run early and often, at every level of the work. Validation depends on the people with the need and usually comes later.

Is testing verification or validation?

Most automated testing is verification: it checks code against expected behaviour someone wrote down. User acceptance testing, usability studies and beta programs are validation, because real users judge whether the product fits their need. A green automated suite is narrower than either: it shows the assertions held, not that the behaviour or the product is right.

What is verification vs validation in software testing?

In software testing, verification covers reviews, static analysis, unit, integration and system tests, which compare the software to its requirements. Validation covers acceptance testing and real use, which compare it to what users need. Verification finds defects against the spec, validation finds a spec that was wrong, and a release needs both.

Is an AI agent's self-report verification or validation?

Neither. A summary saying "done, tests pass" is a claim, written with the same understanding that produced the code, so its blind spots repeat. Verification needs an independent check of what the running app did against the spec. Validation needs a person who knows what users need to judge whether that was the right thing.