AI Verification: Claude Proves Code Safety

Anthropic used its Claude model to mathematically verify that existing software behaves correctly.

The company stated that this approach reduces the need for extensive human review in code verification. The work represents a step toward making AI-generated software easier to trust. However, the proofs have strict limits on the conditions under which they hold.

Traditional formal verification is expensive and slow. Specialists required 20 person-years to verify seL4, a small operating system kernel. This cost has limited the technique to high-stakes areas like Airbus avionics. Anthropic partnered with Theorem to automate this process using Claude. The model generated approximately 180 million lines of mathematical proof code in a matter of weeks. It drew on test programs from 123 open-source libraries in Google's OSS-Fuzz security project.

Claude did not write new software but verified existing code. It turned crash tests into mathematical problems and analyzed compiled machine code. A separate program called Rocq automatically checks these proofs. The researchers described the proof generation as fully autonomous. Human experts built the underlying tools and formal foundations. Claude encountered hundreds of bugs during the process and was permitted to fix them. Many of these bugs likely escaped conventional testing because they appear only under narrow conditions. In one week, the team found as many security bugs in a widely used PDF reader as are usually reported in a full year. Claude also identified a logical error that had gone unnoticed for more than a decade.

Anthropic noted that formally verified does not mean error-free. The proofs hold only under defined conditions. These include a particular processor architecture, single-threaded programs, and inputs smaller than 64 MB. Real-world deployments often run on different hardware and configurations. Individual proofs consume hundreds of millions of tokens. Anthropic has not published total project costs. However, the company argues that verifying AI code could cut review costs. A smaller test run showed that Opus 5.5 solves half the proofs using about 40 percent fewer tokens than Opus 5.

The project shifts the focus from whether AI can program to whether it can prove its code is safe. Anthropic expects formal verification with AI to become as practical as standard testing. Theorem's roadmap targets critical infrastructure including glibc, OpenSSL, and cURL.