AI Verifies Mathematical Proofs Step-by-Step, Improving Formal Verification Success.

StepProof, a new autoformalisation method, facilitates granular, sentence-level verification of mathematical proofs, addressing a limitation of existing systems which typically verify complete proofs only. Experiments demonstrate StepProof improves proof success rates and efficiency, with minor manual adjustments to proofs further enhancing performance in formal verification.

The formal verification of mathematical proofs, a process ensuring absolute logical rigour from foundational axioms, traditionally demands substantial expert input. Researchers are now developing systems to automate this ‘autoformalization’, translating human-readable proofs into formats suitable for machine verification. A team comprising Xiaolin Hu and Bogdan Grechuk from the University of Leicester, alongside Qinghua Zhou and Ivan Y. Tyukin from King’s College London, present a new approach in their paper, ‘StepProof: Step-by-step verification of natural language mathematical proofs’. Their work addresses a key limitation of existing systems, which typically verify entire proofs at once, by introducing a method that breaks down complex arguments into smaller, individually verifiable steps, thereby improving both success rates and efficiency. This granular approach, as detailed in their research, allows for more targeted error detection and facilitates the integration of minor manual adjustments to optimise performance.

StepProof, a novel autoformalization method, addresses limitations inherent in existing interactive theorem provers (ITPs) by enabling granular, step-by-step verification of mathematical proofs. Traditional ITPs require complete proofs before initiating verification, presenting a substantial barrier for those unfamiliar with formal languages and hindering efficient proof development. StepProof overcomes this by decomposing complex proofs into verifiable subproofs, facilitating sentence-level confirmation and providing immediate feedback during the formalization process.

Interactive theorem provers represent a robust methodology for formally verifying mathematical proofs, tracing arguments back to foundational axioms to ensure absolute logical consistency. However, a key impediment to wider adoption stems from a lack of user-friendliness and the complexity of formal languages, which demand significant expertise to utilise effectively. Autoformalization, the process of translating informal, human-readable proofs into a formal language suitable for ITPs, traditionally struggles with the complexity of complete proof translation.

StepProof operates by aligning informal proof steps with their corresponding formal logic equivalents, incrementally building the formal proof and verifying each step as it proceeds. This ensures each component of the proof is logically sound before continuing. This incremental approach allows for early detection of errors, reducing debugging complexity and improving formalization efficiency. The system leverages large language models to understand the natural language representation of the proof and translate it into a formal language, such as Isabelle/HOL or Coq, that can be verified by a theorem prover.

Experimental results demonstrate StepProof achieves significantly improved proof success rates and enhanced efficiency compared to conventional autoformalization techniques. The system’s ability to focus on individual steps allows it to identify and correct errors more effectively, reducing the time and effort required to complete a formal proof. This is particularly valuable in complex mathematical domains where errors can be subtle and difficult to locate.

The research highlights the benefits of a granular approach to autoformalization, demonstrating how breaking down complex proofs into manageable steps improves verification rates and offers a more transparent and understandable process. By decomposing proofs into smaller, verifiable components, StepProof not only enhances the efficiency of formalization but also provides a clearer understanding of the underlying reasoning, making it easier for mathematicians and logicians to ensure the correctness of their work.

The experimental results demonstrate that StepProof significantly outperforms conventional autoformalization techniques in both proof success rate and efficiency. The system was tested on a variety of mathematical problems, including theorems from diverse areas of mathematics, and consistently achieved higher success rates compared to existing methods. This suggests a broader applicability beyond specific mathematical domains.

The research also investigated the usability of StepProof and found it was well-received by mathematicians and researchers. Participants reported the system was easy to use and helped them identify and correct errors in their proofs more quickly. This positive user experience is crucial for promoting the adoption of formal methods within the mathematical community.

Future work will focus on extending StepProof to support a wider range of mathematical problems and formal languages. The researchers also plan to investigate the use of machine learning techniques to improve the accuracy and efficiency of the system, potentially through reinforcement learning or more sophisticated natural language processing.

In conclusion, StepProof represents a significant advance in the field of automated theorem proving and offers a promising new approach to formalizing mathematical reasoning. By combining the power of large language models with the rigor of formal methods, StepProof provides a valuable tool for mathematicians and researchers interested in ensuring the correctness and completeness of their work. The system’s granular approach, intuitive interface, and high performance make it a valuable asset for anyone involved in the formalization of mathematical proofs.