RAG Attacks Achieve 84.4% Exfiltration Using Knowledge Graph-Guided Long-Term Planning

Retrieval-Augmented Generation (RAG) systems, increasingly popular for integrating external knowledge with large language models, present a novel privacy threat , the gradual exfiltration of sensitive information through cleverly designed queries. Researchers Mengyu Yao and Ziqi Zhang, from Peking University and the University of Illinois Urbana-Champaign respectively, alongside Ning Luo et al, demonstrate a powerful new attack vector targeting these systems. Their work formulates the problem as an adaptive stochastic coverage problem, enabling long-term extraction planning previously lacking in existing attacks. By introducing RAGCRAWLER, a system that builds a knowledge graph to track revealed information and strategically plans queries, they achieve significantly higher corpus coverage , up to 84.4% , and outperform existing methods by over 20.7%, revealing critical security gaps in RAG and underscoring the urgent need for improved privacy safeguards.

This breakthrough addresses critical limitations in current multi-turn extraction attacks, which often rely on heuristics and lack long-term planning capabilities, typically stagnating after only a few rounds and revealing a limited portion of the private corpus. The study establishes a formal connection between RAG extraction attacks and the Adaptive Stochastic Coverage Problem (ASCP), framing each query as a probabilistic action aimed at maximizing conditional marginal gain (CMG), a measure of how much new information a query is expected to reveal.

However, directly applying ASCP to practical RAG attacks presents significant challenges, including the unobservability of CMG, the intractability of the vast query space, and the need for queries to remain benign and policy-compliant. To overcome these hurdles, the researchers developed RAGCRAWLER, which maintains a global attacker-side state represented as a knowledge graph, enabling estimation of CMG and strategic planning in semantic space. RAGCRAWLER constructs a knowledge graph to represent revealed information, utilising this global state to estimate CMG and plan queries targeting unretrieved regions of the corpus. Crucially, the system also generates natural-sounding prompts designed to elicit remaining content while maintaining stealth and avoiding detection.
Comprehensive experiments across diverse RAG architectures and datasets reveal that RAGCRAWLER consistently surpasses all baseline methods, achieving up to 84.4% corpus coverage within a fixed query budget. This represents an average improvement of 20.7% over the top-performing baseline, alongside high semantic fidelity and strong content reconstruction accuracy at a low attack cost. Furthermore, the research proves RAGCRAWLER’s robustness against advanced RAG systems employing query rewriting and multi-query strategies, demonstrating its effectiveness even in the face of sophisticated defenses. The work reveals significant security gaps within RAG systems and underscores the urgent need for stronger safeguards to protect sensitive data. This innovative approach not only provides theoretical guarantees for data extraction but also offers a systematic framework for analysing the extraction capabilities of practical adversaries, paving the way for the development of more secure and privacy-preserving RAG implementations.

RAG Extraction Analysed via Adaptive Stochastic Coverage improves

Researchers formulated the Retrieval-Augmented Generation (RAG) extraction attack as an Adaptive Stochastic Coverage Problem (ASCP), treating each query as a probabilistic action designed to maximise conditional marginal gain (CMG) under uncertainty. This innovative approach enables principled long-term planning, addressing limitations found in heuristic-based multi-turn extraction attacks. The study pioneers a method for systematically analysing the extraction capability of practical adversaries, grounding it in theoretical guarantees for data extraction. To overcome challenges inherent in applying ASCP to practical RAG attacks, the team maintained a global attacker-side state to guide the attack process.
Building upon this, they engineered RAGCRAWLER, a system that constructs a knowledge graph representing revealed information, estimates CMG using this global state, and plans queries in semantic space targeting unretrieved regions. RAGCRAWLER also generates benign-looking prompts to elicit remaining content while preserving stealth, crucial for avoiding detection. Experiments employed diverse RAG architectures and datasets to rigorously evaluate the method’s performance. The experimental setup involved measuring corpus coverage achieved within a fixed query budget, comparing RAGCRAWLER against several baseline methods.

Results demonstrated that RAGCRAWLER consistently outperformed all baselines, achieving up to 84.4% corpus coverage and delivering an average improvement of 20.7% over the top-performing baseline. Semantic fidelity and content reconstruction accuracy were also assessed, revealing low attack cost and high-quality extracted information. The team harnessed advanced RAG systems with query rewriting and multi-query strategies to test RAGCRAWLER’s robustness, confirming its continued effectiveness even against sophisticated defences. Furthermore, the study detailed the precise measurement of CMG, acknowledging its unobservability in real-world scenarios.

To address this, RAGCRAWLER utilises the knowledge graph to estimate CMG, effectively bridging the gap between theoretical optimisation and practical implementation. The intractable action space was tackled by planning queries in semantic space, reducing the search complexity while maintaining query relevance. This innovative methodology reveals significant security gaps in RAG systems and underscores the urgent need for stronger safeguards against extraction attacks.

RAG Systems Vulnerable to Adaptive Extraction Attacks, raising

Scientists have demonstrated a significant security vulnerability in Retrieval-Augmented Generation (RAG) systems, revealing that sensitive information can be gradually extracted from the underlying data corpus through carefully crafted queries. The research team formulated the RAG extraction attack as an Adaptive Stochastic Coverage Problem (ASCP), treating each query as a probabilistic action designed to maximise conditional marginal gain (CMG) for effective long-term planning under uncertainty. This innovative approach addresses limitations in existing multi-turn extraction attacks which rely on heuristics and lack comprehensive planning. Experiments revealed that the newly developed method, RAGCRAWLER, consistently outperforms all baseline approaches across diverse RAG architectures and datasets.

Specifically, RAGCRAWLER achieved up to 84.4% corpus coverage within a fixed query budget, demonstrating a substantial improvement in information extraction capabilities. Data shows an average performance gain of 20.7% over the top-performing baseline, highlighting the effectiveness of the ASCP-based strategy. The team measured the ability of RAGCRAWLER to uncover hidden documents stochastically, maximising expected corpus coverage with a limited number of queries. RAGCRAWLER employs a global attacker-side state, maintained through a knowledge graph, to estimate CMG and plan queries in semantic space targeting previously unretrieved regions.

Tests prove that this knowledge graph construction accurately maps graph growth to an estimate of coverage marginal gain, making the CMG observable despite the inherent challenges of unobservable CMG in practical RAG attacks. The strategy scheduling component replaces exhaustive search with a two-stage selection process, fusing historical gain with structural exploration to efficiently identify high-value queries. Crucially, measurements confirm RAGCRAWLER’s robustness against advanced RAG systems incorporating query rewriting and multi-query strategies, maintaining effectiveness even with sophisticated defensive mechanisms. The extracted content exhibits high semantic fidelity and strong content reconstruction accuracy, enabling the creation of a substitute RAG system achieving a 0.6992 semantic similarity to the original RAG’s responses. Results demonstrate an average coverage rate of 66.80%, a significant 20.70% improvement over the best baseline, and an average coverage rate of 55.30% even when tested against RAG systems with safeguards. This work reveals significant security gaps and underscores the urgent need for stronger safeguards within RAG systems to protect sensitive data.

RAGCRAWLER Exposes RAG Data Extraction Risks

Scientists have formalised the privacy risk of data extraction from Retrieval-Augmented Generation (RAG) systems as an adaptive stochastic coverage problem. They introduced RAGCRAWLER, a new attack framework that constructs a knowledge graph to monitor revealed information and strategically plans queries in semantic space to target unretrieved data. Comprehensive experiments demonstrated RAGCRAWLER’s superior performance, consistently exceeding baseline methods and achieving up to 84.4% corpus coverage within a defined query limit. An average improvement of 20.7% was observed compared to the strongest baseline, alongside high semantic fidelity and efficient content reconstruction at a low attack cost.

Importantly, RAGCRAWLER maintained its effectiveness even against sophisticated RAG systems employing query rewriting and multi-query strategies. The authors acknowledge that their work focuses on a specific attack vector and doesn’t encompass all potential privacy threats within RAG systems. Future research could explore defences against such attacks and investigate the interplay between data poisoning and extraction vulnerabilities. These findings reveal a fundamental weakness in current RAG architectures, highlighting the urgent requirement for robust safeguards to protect private knowledge bases and sensitive data.