Claude Turns Software Tests into Machine-Checked Proofs
formal verification（formal verification）
Checking code with mathematics against a clearly stated rule.
fuzz testing（fuzz testing）
Trying many changed inputs to find program failures.
Rocq（Rocq）
A system that checks whether a mathematical proof follows its rules.
What happened
Anthropic, the AI company behind Claude, published a report on October 22, 2026. Working with Theorem, a software-verification company, Anthropic says Claude produced formal proofs for security properties in 123 open-source libraries. The libraries were compiled for RISC-V, a processor design that defines machine instructions. The project covered 239 test entry points and ran for a few weeks. The original report describes the work.
The story received 164 points and 79 comments on Hacker News. Those numbers show community attention. They do not prove the report is correct.
From tests to proofs
Fuzz testing sends many changed or random inputs into a program. It can find crashes, but it cannot try every possible input. Anthropic’s team converted existing fuzz tests into a formal rule. The rule said a program should not crash for any input below 64 MB, under a specific machine model. Claude wrote the proof. Rocq, a proof-checking system, checked it.
This changes the question from Did the program survive these examples? to Does it satisfy this rule for every allowed input? The second question is stronger. It is also limited by the rule and the model used to define it.
Why it matters
Formal verification has traditionally required unusual expertise and long projects. Proofs can be much larger than the code they examine. Anthropic cites one example where implementing an operating-system kernel took two person-years, while formally verifying it took 20 person-years.
If language models can handle more routine proof writing, verification could reach more production software. That is a possible consequence, not a result proven by this experiment. People would still need to define the properties that matter. They would also need to check whether the model matches the real machine.
What the experiment showed
According to Anthropic, Claude worked across 123 OSS-Fuzz libraries and 239 entry points. The resulting proofs contained about 180 million lines of Rocq code. A typical proof had about 600,000 lines. During the process, Claude found hundreds of bugs, patched some of them, and continued proving the revised code.
The report also describes other case studies. Claude helped prove properties of the zlib implementation. In another experiment, a Rust Python interpreter matched CPython on 99.95% of 1.7 million randomly generated programs. These results broaden the story, but they remain experiments reported by Anthropic.
What it did not show
Formally verified does not mean bug-free. The main proof assumed a single-threaded, statically linked RISC-V program, a fixed memory budget, and modeled Linux behavior. Many real systems use different processors or operating-system setups. The proof covered selected entry points, not every possible use of each library.
The report also says that a proof checker cannot decide whether the formal specification matches reality. The trusted pieces include the machine models, the theorem statements, and the checker itself. Problems outside those pieces can remain.
What to watch next
The approach can require hundreds of millions of tokens, so cost matters. Anthropic reports that a newer model used about 40% fewer tokens on half of 36 small tasks. A shared workspace also reduced token use by 17% in its comparison.
The next tests are practical. Can other researchers verify the released proofs? Can the models cover more processors and operating systems? Can proofs be reused across software projects? The important claim is not that AI has solved software correctness. It is that machine-checked guarantees may become much easier to produce than before.
Claude Helps Prove Software Rules
📰 Full story: Claude Turns Software Tests into Machine-Checked Proofs
Claude helped turn software tests into math-based rules for 123 libraries.
formal verification（formal verification）
Using math to check whether code follows a clear rule.
fuzz test（fuzz test）
A test that sends many changed inputs into software.
RISC-V（risk five）
A design that tells a processor which instructions to run.
💡 The gist
- Claude helped prove safety rules for 123 software libraries.
- The work used RISC-V programs and 239 test entry points.
- The proof covered special conditions, not every possible use.
Anthropic, the AI company behind Claude, published the report on October 22, 2026. It worked with Theorem, a software-verification company. The original report explains the project.
A normal test checks many examples. A fuzz test creates many changed inputs. It looks for crashes or other failures. But no test can try every possible input.
The team changed existing tests into a stronger rule. The rule covered inputs smaller than 64 MB. It said the program should not crash under certain conditions. Claude wrote a proof for the rule. Rocq checked the proof with math.
This is called formal verification. It checks whether code follows a clear rule. A proof can cover every allowed input. A test can only cover the inputs it tried. That makes proofs useful for finding rare problems.
Claude found hundreds of bugs during the project. It patched some code and tried the proof again. The final proofs contained about 180 million lines. That shows how much work the AI helped organize.
The result still has limits. The main proof used RISC-V, a particular processor design. It also assumed a specific Linux model. It covered selected test entry points. It did not cover every library behavior or every computer.
The Hacker News post received 164 points and 79 comments. Those numbers show interest from a technology community. They do not prove the research is correct.
The next question is cost. Proofs can need hundreds of millions of tokens. Researchers also need to test more processors and operating systems. They should check the released proofs independently. They must also make sure each rule matches real software use.
The project does not show that AI makes all software safe. It shows a possible new workflow. AI writes much of the proof. A checker confirms whether the proof follows the rules. Read more in the Anthropic report.
💬 Why scaling LLM proofs is harder than making one proof
The main thread is that an LLM producing one proof is different from keeping many proofs useful in a large software system.
- Creating genuinely new mathematics and checking lots of existing software are different tasks. An LLM might be useful for the second task even if it cannot go far beyond human mathematical knowledge.
- LLM proofs often explain too many small details. Large programs need reusable abstractions and modules. Anecdotal users also report duplicated, fragile LLM code and tests, although some commenters hope reinforcement learning can improve the strategy.
- Proof writing is closely related to programming through Curry–Howard. Better proof skills might help coding, but the connection does not guarantee better abstraction or structure.
- A formal proof can cover every input in a modeled range, while fuzzing checks only samples. But the result is only as good as the property and specification being proved.
- The model must also match the real program and its compiler or hardware assumptions. Declarative languages may help, but writing precise requirements and building efficient compilers remain hard.
initial digest at 274 comments (revision 4). We fetched 100 comments and sampled 100 across the thread. These are HN users’ reports, not independently verified facts.
Claude Checks Computer Promises
📰 Full story: Claude Turns Software Tests into Machine-Checked Proofs
Claude used math to check whether computer programs kept their rules.
Claude（Claude）
An AI helper made by Anthropic.
Rocq（Rocq）
A tool that checks a math promise.
Hacker News（Hacker News）
A website where people discuss technology.
Anthropic, the company that makes Claude, asked Claude to inspect computer instructions.
Computer programs have rules.
Claude looked for broken rules.
It wrote a math promise about the program.
Rocq, a checking tool, checked that promise.
The project studied 123 software libraries.
It also found some broken parts.
People fixed some of those parts.
But this did not prove every program was safe.
The promise worked only under special conditions.
Hacker News, a technology discussion website, showed 164 points and 79 comments.
Those numbers show interest.
They do not show truth.
Read the Anthropic report for the full story.
💬 Can AI check lots of programs?
AI may help check programs, but making the checks work for very big programs is difficult.
- Making new math is hard. Checking many programs that already exist is a different job, and AI might be useful there.
- AI proofs can be much too detailed. Big programs need short rules that group similar things together. Better training might help, but nobody has proved that yet.
- Trying a few inputs is like fuzzing. A formal proof can sometimes check every input in a chosen range. But if we ask the wrong question, the proof can still be about the wrong thing.
- We must also check that the model, the real program, and the compiler agree. A simpler programming language might help, but making it precise and fast is still hard.
initial digest at 274 comments (revision 4). We fetched 100 comments and sampled 100 across the thread. These are HN users’ reports, not independently verified facts.
💬 For LLM verification, the hard problem is scaling proofs, not producing one
HN commenters see promise in machine-checked proofs, but argue that specifications, abstractions, and model boundaries may matter more than the first successful proof. They do not agree on whether training can overcome the scaling wall.
initial digest at 274 comments (revision 4). We fetched 100 comments and sampled 100 across the thread. These are HN users’ reports, not independently verified facts.