Anthropic and Theorem automate software safety proofs

Anthropic has announced a significant advancement in software reliability by utilizing its Claude model to automatically generate formal proofs of code correctness. Working alongside partner Theorem, the company demonstrated that artificial intelligence can now handle the rigorous mathematical verification of existing software, a task that has historically required specialized human expertise. This development marks a shift from merely generating code to certifying its safety through mathematical logic.

The scale of the operation was substantial. In a period of just a few weeks, the system produced approximately 180 million lines of proof code. These proofs were derived from test programs associated with 123 open-source libraries within Google's OSS-Fuzz security project. The process involved translating crash tests into mathematical problems and analyzing compiled machine code. An independent verification program named Rocq then checked these proofs, a workflow that Anthropic describes as requiring minimal human review.

The team behind this effort includes Anthropic's Frontier Red Team and its partner, Theorem. While the human researchers at Theorem built the underlying formal foundations and tools, the actual proof generation was executed autonomously by the model. This division of labor allowed the AI to operate at a scale that would be difficult for a human team to match within a short timeframe. The result is a system that can independently validate software integrity without constant human intervention.

One of the most striking outcomes of this process was the discovery of numerous defects. The system identified hundreds of bugs that conventional testing methods had failed to catch. Many of these issues only manifested under specific conditions, such as unusually large input files or particular sequences of data access. In one notable instance, the verification of a widely used PDF reader yielded as many security bugs in a single week as are typically reported for that software over an entire year. Additionally, a logical error that had remained undetected for more than a decade was uncovered during the process.

Despite these successes, the approach comes with specific limitations. Anthropic clarifies that formal verification does not guarantee that software is entirely error-free. The proofs are valid only under strict conditions, including a specific processor architecture, single-threaded programs, and input sizes under 64 MB. The computational cost is also high, with individual proofs consuming hundreds of millions of tokens. However, data suggests that newer models are becoming more efficient, with Opus 5.5 solving half the proofs using roughly 40 percent fewer tokens than its predecessor, Opus 5.

The broader implication of this technology is a potential transformation in how software is developed and trusted. If AI can both write code and prove its safety, the industry may move toward a model where formal verification becomes as routine as standard testing. Theorem's roadmap already targets critical infrastructure components such as glibc, OpenSSL, and cURL. This progression suggests that the question in software development is shifting from whether AI can program to whether it can guarantee the safety of that code, a change that could have far-reaching consequences for the tech industry.

This milestone highlights the growing role of AI in ensuring software quality. By automating the proof generation process, developers may be able to trust AI-generated software more confidently, reducing the time spent on manual review. As the technology matures, the combination of code generation and automated verification could become a standard part of the development lifecycle, particularly for mission-critical systems where reliability is paramount.