What It Actually Means to Mathematically Prove Code Is Correct
Mistral’s Leanstral 1.5 Uses a Formal Proof Assistant to Verify AI-Generated Code Behaves as Intended, Instead of Just Testing It and Hoping the Test Cases Were Good Enough. Here Is Why That Distinction Matters More Now Than It Ever Has.
Mistral released Leanstral 1.5 this week, a model that goes beyond generating code to providing mathematical proof, via the Lean 4 proof assistant, that the code behaves as intended. This is not a marginally better test suite. It is a fundamentally different way of establishing correctness — the same deterministic-tool logic this series covered on Day 33 for AI biology, now applied to software verification. Day 37 explains what a formal proof actually establishes that testing cannot, and why it matters specifically now that so much production code is AI-generated in the first place.
A test suite tells you the code worked for every case someone thought to write a test for. A formal proof tells you the code works for every case, full stop, including the ones nobody thought of. That is not a difference of degree. It is a difference between confidence and certainty, and until this week that certainty was almost never available for AI-generated software.
Neal Lloyd · Inside The Machine, Day 37Day 33 of this series covered how routing Claude to deterministic computational tools, rather than letting it answer from language alone, moved AI biology accuracy from 16.9% to 92.8%. Mistral’s Leanstral 1.5, released this week, applies the identical underlying logic to a different domain: instead of generating code and hoping test coverage is adequate, it produces a formal mathematical proof, checked by the Lean 4 proof assistant, that the code satisfies its specification — a verification standard categorically stronger than passing tests. This is Day 37 of Inside The Machine. Today we explain what a formal proof actually establishes, why it is different in kind from testing rather than just more of it, and why this matters more now that a majority of production code at some companies is already AI-generated.
A Passing Test Suite Proves the Code Works for the Cases You Thought Of
Software testing works by example: write a set of specific inputs, check that the code produces the expected output for each one, and treat a fully passing suite as evidence the code is correct. That evidence is real and valuable, but it is fundamentally bounded by the imagination of whoever wrote the tests. A test suite cannot catch a bug in an input combination nobody thought to write a test for — and for any function complex enough to matter, the space of possible inputs is almost always larger than the space of inputs anyone actually tested.
AI-generated code sharpens this problem rather than solving it. A model can generate code that passes every test in a given suite while still containing a genuine logical error in an edge case the test suite never covered — and unlike a human engineer who wrote the function and has some intuitive sense of where its blind spots are, a reviewer evaluating AI-generated code often has no equivalent intuition to draw on, because they did not build the mental model of the function’s behaviour that writing it from scratch would have produced.
Formal verification takes a categorically different approach: instead of checking specific example inputs, it constructs a mathematical proof that the code satisfies a given specification for every possible input the specification covers, not just the ones someone thought to test. Lean 4, the proof assistant Leanstral 1.5 uses, is a system originally built for verifying mathematical theorems, repurposed here to verify that a piece of software’s actual behaviour matches its intended specification with the same rigour a mathematician would demand of a theorem’s proof.
Testing: run the code against a chosen set of inputs, confirm the outputs match expectations for those inputs specifically. Strong evidence, bounded by what the test author thought to check. Formal verification: construct a mathematical proof that the code satisfies its specification for every input the specification covers. Categorically stronger, and until recently, far more labour-intensive to produce for anything beyond small, well-isolated functions.
The Same Move That Fixed AI Biology, Now Fixing AI Software
The parallel to Day 33 is close to exact. There, the problem was that a language model answering a biology question from memory alone produces plausible-sounding text that is often subtly wrong, because the model is optimised to generate text that looks correct rather than to compute an answer that is verified correct. Routing the question to a deterministic structural biology solver instead closed that gap, because the tool computes rather than recalls. Here, the problem is that a language model generating code produces plausible-looking code that may be subtly wrong in ways a test suite does not catch, for exactly the same underlying reason: the model is optimised to produce output that looks like a correct solution, not output verified against a formal specification.
Leanstral 1.5’s approach routes the verification step to Lean 4 rather than asking the model to somehow know its own code is correct. The model generates both the code and a proposed formal specification of what that code should do, and Lean 4 then either verifies that the code provably satisfies the specification or rejects the proof, forcing revision. This is the code equivalent of Claude Science routing a fold-prediction question to a structural biology solver instead of guessing — delegate the part of the task that requires actual verified computation to a system built for exactly that, rather than asking a language model to bluff its way through it.
Mistral’s reported benchmark results show significant progress specifically on formal software verification for critical systems — the category of software where a subtle logic error has outsized consequences: financial infrastructure, safety-critical embedded systems, cryptographic implementations. That is precisely where the gap between passing tests and being provably correct matters most, and precisely where AI-generated code has, until now, carried a trust deficit that testing alone could not fully close.
Deterministic tools keep showing up as the actual fix, once you look closely enough at any AI accuracy problem worth solving. Biology got one on Day 33. Software just got one this week. The pattern is not a coincidence — it is what happens whenever a domain has a real, checkable notion of correct that language alone cannot reliably produce.Neal Lloyd · Inside The Machine, Day 37
A Proof Is Only as Good as the Specification It Proves
Formal verification is not a magic guarantee that a piece of software is bug-free in every sense a user might care about. A proof only establishes that code satisfies its specification — and if the specification itself is wrong, incomplete, or fails to capture a requirement the developer never thought to formalise, a perfectly valid mathematical proof will happily verify code that still does not do what anyone actually wanted. Writing a correct, complete specification is itself a genuinely hard problem, and arguably the harder half of what formal verification actually requires.
That limitation is not a reason to dismiss the approach; it is a reason to be precise about what it proves. Leanstral 1.5 verifying that generated code matches its specification is a categorically stronger claim than a test suite passing, and a meaningful step forward for AI-generated code in domains where correctness genuinely matters. It is not, on its own, a claim that the specification captured everything a human stakeholder actually cared about — that remains a human judgment call, made before the formal verification step ever runs.
The practical upshot for anyone evaluating AI-generated code going forward: a formal proof of correctness against a specification is a much stronger signal than passing tests, and worth actively seeking out for safety- or security-critical code specifically. But asking what the specification actually says, and whether it captures the requirement you actually care about, remains the step no proof assistant can do for you.
A proof cannot save you from a bad specification. It can only guarantee the code matches whatever specification you actually wrote down, which means the hardest and most human part of formal verification was never the proving. It was deciding, precisely and completely, what correct actually means before you ask a machine to check it.Neal Lloyd · Inside The Machine, Day 37
Inside The Machine, Day 37 · July 18 2026
Neal Lloyd writes about technology, human adaptation, and the uncomfortable questions nobody wants to answer at dinner. Inside The Machine is his ongoing daily series on AI.
- Day 01What Is This Thing?
- Day 02Survive the Machine
- Day 03The Great Debate
- Day 04Who Gets Hurt?
- Day 05Who’s In Charge?
- Day 06The Industries That Win
- Day 07The Human Edge
- Day 08The Creativity Question
- Day 09Does AI Feel Anything?
- Day 10The Data Problem
- Day 11The Trust Question
- Day 12The Accountability Gap
- Day 13The Rewired Brain
- Day 14Open vs Closed
- Day 15The New Cold War
- Day 16Why AI Lies With Confidence
- Day 17AI Is Eating the Power Grid
- Day 18The Age of AI Agents
- Day 19AI Safety Was Never Just Theory
- Day 20The Surveillance Question
- Day 21AI and the Future of Education
- Day 22AI and Your Health
- Day 23What Is AGI and Are We Close?
- Day 24What Is Work For?
- Day 25AI and Democracy
- Day 26AI and the Future of Money
- Day 27Can the Planet Afford AI?
- Day 28Why AI Forgets Everything
- Day 29Can Anyone Actually Govern AI Now?
- Day 30Inside the Jailbreak Severity Framework
- Day 31Why No One Can Guarantee Your AI Agent Will Do What It Was Told
- Day 32Squidbleed: A 29-Year-Old Bug and the Same Capability That Got Fable 5 Recalled
- Day 33How Deterministic Tools Took AI Biology From 16.9% to 92.8%
- Day 34The Three Risks Hiding Inside Your Enterprise’s Chinese AI Traffic
- Day 35The LiteLLM Vulnerability Is a Preview of Gateway Security Done Wrong
- Day 36Why You Cannot Patch an Anti-Addiction Law Onto a Memory-Persistent Companion
- Day 37What It Actually Means to Mathematically Prove Code Is CorrectYou are here
