AI Code Generation Tools Repeat Security Flaws, Creating Predictable Software Weaknesses

Researchers are increasingly concerned with predictable vulnerabilities arising from the use of large language models (LLMs) for software generation. Tomer Kordonsky, Maayan Yamin, and Noam Benzimra, all from the Technion, Israel Institute of Technology, alongside Amit LeVi from both Technion and Zenity, and Avi Mendelson from Technion, present a novel approach to understanding and exploiting this phenomenon, termed ‘vulnerability persistence’. Their work introduces the Feature, Security Table (FSTab), a method for predicting backend vulnerabilities from observable frontend features, even without access to the underlying code or model details. Significantly, FSTab also quantifies the consistency with which LLMs reproduce vulnerabilities across different programs and domains, demonstrating strong cross-domain transfer with up to 94% attack success on models such as Claude-4.5 Opus. These results reveal a previously unexamined attack vector in LLM-generated software and underscore the potential security implications of relying on these models for code creation.

This work addresses a critical gap in LLM security, moving beyond post-hoc detection to proactively predict potential weaknesses based on observable characteristics of the generated code.

FSTab functions as both a black-box attack mechanism and a model-centric evaluation framework, revealing how consistently specific vulnerabilities recur across different programs, rephrased prompts, and application areas. The core innovation lies in linking visible frontend features of software with likely backend vulnerabilities, all without requiring access to the underlying source code or model internals.
This research demonstrates that LLMs often rely on recurring templates during code generation, creating predictable patterns that can be exploited. This model-centric evaluation allows for direct comparison of LLMs as code generators, highlighting their inherent security profiles. The findings expose an underexplored attack surface in LLM-generated software, emphasizing the risks associated with relying on these models for code creation and the need for proactive security measures. FSTab functions as a lookup structure mapping backend vulnerabilities to visible frontend features for a specific source model, facilitating the automated black-box attack.

Given a website and the name of the generating LLM, the attack scans visible features and uses FSTab to infer potential vulnerabilities in the hidden software components without requiring access to the source code. This process prioritizes likely weaknesses, enabling efficient vulnerability triage in black-box settings.

Beyond the attack, the research introduced a framework for quantifying vulnerability recurrence across domains and features, employing metrics such as Feature-Vulnerability Rate, Recurrence Vulnerability Profile, Domain Vulnerability Rate, and Cross-Domain Transfer. The evaluation involved testing the models across six code LLMs and five diverse application domains, revealing that vulnerability patterns can be both persistent and model-specific. The study quantifies vulnerability recurrence across programs, semantics-preserving rephrasings, and application domains through a model-centric evaluation framework.

This framework utilizes FSTab, a lookup structure mapping backend vulnerabilities to visible frontend features for a given source model, enabling black-box vulnerability prioritization. Construction of FSTab involved generating large corpora from target models and labeling them with vulnerability detectors and feature scanners.

Analysis reveals that vulnerability patterns exhibit persistence and model specificity across diverse application domains, exposing a previously under-explored attack surface in LLM-generated software. The work proposes an automated black-box attack leveraging FSTab to infer hidden vulnerabilities from visible functionality without requiring access to source code.

This approach allows for triage of likely weaknesses based solely on model identity and observable features. The research establishes a new threat model where attackers can exploit predictable insecure code generation at scale, increasing the risk of widespread cyber attacks.

Consistent Vulnerability Transfer Across Language Models Demonstrated by FSTab

Predictable patterns in the outputs of large language models (LLMs) create recurring vulnerabilities in the software they generate. The research identified confirmed vulnerabilities including denial-of-service attacks via regular expression denial of service, the absence of rate limiting on login attempts, and potential injection flaws.

The authors acknowledge that some vulnerability impacts were limited by the testing environment, preventing full confirmation in all cases. They also intentionally omitted specific exploit details to avoid creating an operational exploitation manual, focusing instead on demonstrating the reduction in attacker search cost. Future work could explore methods for mitigating these vulnerabilities at the LLM generation stage, potentially through techniques that encourage more diverse and unpredictable outputs.