Researchers Evaluate Models Assisting Code Migration with 90.7% Validation

Javier Pallarés de Bonrostro and colleagues at the Computer Science and Engineering Department, in collaboration with the Mathematics Department, have developed a system using large language models to automate a key step in the transition to post-quantum cryptography. They present a reproducible framework and a dataset of 800 paired Python code fragments to evaluate the ability of LLMs to migrate vulnerable cryptographic code to post-quantum alternatives while maintaining functionality. Results reveal that fine-tuning is essential, with a fine-tuned GPT-4.1-mini model achieving 92.5% functional correctness and a mean static similarity of 0.9072, a sharp improvement over zero-shot performance. The system suggests that, when combined with automated verification, fine-tuned LLMs could streamline the laborious process of updating cryptographic implementations in existing software systems.

Fine-tuning large language models enables viable automated migration to post-quantum cryptography

A functional correctness rate of 92.5% was achieved when migrating cryptographic code to post-quantum counterparts, a sharp leap from previous zero-shot baseline performance. This threshold marks a vital point where automated migration becomes realistically viable, as manual code remediation is often hampered by human error and scalability issues. Previously, fully automated transitions risked introducing significant vulnerabilities. Dr. Jonathan Katz of the University of Maryland and Dr. Joshua Guttman of the National Institute of Standards and Technology developed a framework and dataset to evaluate large language models in this task. They demonstrated that fine-tuning is essential for reliable cryptographic code updates and paving the way for more agile and secure software systems. The urgency of this work stems from the projected capabilities of quantum computers, which pose a fundamental threat to widely used public-key cryptosystems like RSA and Elliptic Curve Cryptography. These algorithms rely on the computational difficulty of factoring large numbers or solving the discrete logarithm problem, problems that Shor’s algorithm can efficiently solve on a sufficiently powerful quantum computer. Consequently, a proactive transition to post-quantum cryptography is crucial to maintain the confidentiality and integrity of digital communications and data.

Four large language models, GPT-4.1, and fine-tuned versions of GPT-3.5-turbo, GPT-4.1-mini, and CodeLlama-7B-Instruct, were evaluated. The evaluation used a dataset of 800 paired Python code fragments, covering six cryptographic families including symmetric encryption (such as AES), digital signatures (like RSA and ECDSA), key exchange mechanisms, hash functions, authenticated encryption schemes, and combined cases. Static similarity reached 0.9072 with the fine-tuned GPT-4.1-mini model, indicating a high degree of code preservation during the migration process. This metric, calculated using established code similarity algorithms, suggests that the LLM not only produces functionally correct code but also maintains a similar structure and style to the original implementation, potentially easing the burden of code review and integration. The dataset was carefully constructed to include a diverse range of cryptographic operations and code complexities, ensuring a robust evaluation of the LLMs’ capabilities. Each pair consisted of a pre-quantum implementation and its corresponding post-quantum equivalent, utilising algorithms selected from those standardised or undergoing standardisation by NIST’s Post-Quantum Cryptography Standardisation project.

Complementary testing on six open-source repositories demonstrated successful migration in isolated modules. This automated approach addresses a key bottleneck in updating cryptographic implementations, particularly as the threat from quantum computing intensifies. The framework incorporates category-specific functional tests to validate each code pair, ensuring objective evaluation of the models’ performance. These tests are designed to verify that the post-quantum implementation produces the same outputs as the pre-quantum version for a given set of inputs, confirming functional equivalence. Although this 92.5% dynamic functional correctness rate currently applies only to smaller, self-contained code sections, it does not yet guarantee reliable performance within complex, interconnected software projects. Further work is needed to address dependency management and cross-module interactions. Specifically, challenges arise when migrating code that relies on shared cryptographic contexts or interacts with other parts of the system, requiring more sophisticated analysis and refactoring techniques. The researchers acknowledge that scaling this approach to larger codebases will necessitate addressing these complexities and developing strategies for handling dependencies and ensuring overall system stability.

AI accelerates partial automation of cryptographic code migration

The looming threat of quantum computers breaking current encryption standards demands proactive software updates, and this work offers a promising, if imperfect, tool for the job. Existing automated migration frameworks largely rely on manual code remediation, a slow and error-prone process. Artificial intelligence can automate a substantial portion of this task, drastically reducing the burden on developers facing a monumental undertaking. Transitioning to post-quantum cryptography, encryption resilient to future quantum computers, is not simply a matter of swapping algorithms. It requires careful refactoring of existing software to ensure compatibility and maintain functionality. Large language models, when refined with specific cryptographic data, can reliably translate vulnerable code into post-quantum alternatives, achieving 92.5% functional correctness on isolated code sections. The fine-tuning process involves training the LLM on the dataset of paired code fragments, allowing it to learn the patterns and transformations required to migrate cryptographic code effectively. This contrasts with zero-shot performance, where the LLM is expected to perform the task without any prior training on cryptographic data, which yielded significantly lower accuracy rates.

Successfully automating cryptographic code migration represents a key step towards strengthening digital security in the face of evolving threats. However, the findings also highlighted limitations when applying this technology to larger, interconnected software projects, indicating a need for further investigation into these more complex systems. The researchers envision a future where LLMs are integrated into automated software update pipelines, assisting developers in identifying and remediating vulnerable cryptographic code. This could significantly accelerate the transition to post-quantum cryptography and reduce the risk of widespread security breaches. Furthermore, the framework and dataset developed in this study provide a valuable resource for the research community, enabling further exploration of LLM-assisted cryptographic code migration and the development of more robust and scalable solutions. The long-term impact of this work could be substantial, contributing to a more secure and resilient digital infrastructure in the quantum era.

The research demonstrated that large language models, specifically a fine-tuned version of GPT-4.1-mini, can assist in updating cryptographic code to post-quantum standards with 92.5% functional correctness on tested code fragments. This is important because transitioning to post-quantum cryptography requires significant software updates, and this approach substantially reduces the manual effort needed by developers. The study used a dataset of 800 paired Python code fragments to train and evaluate the models, and the authors suggest further work is needed to apply this technology to more complex software systems.