Shows GrepRAG Unlocks Repository-Level Code Completion with Index-Free Retrieval for LLMs

Researchers are tackling the persistent challenge of code completion across entire software repositories, a task complicated by dependencies spanning multiple files and the limitations of current large language model (LLM) context windows. Baoyi Wang, Xingliang Wang, and Guochang Li, all from Zhejiang University, alongside Chen Zhi and Junxiao Han from Hangzhou City University and Xinkui Zhao from Zhejiang University, present a novel investigation into the potential of surprisingly simple retrieval methods. Their work challenges the reliance on computationally expensive semantic indexing and graph analysis by exploring ‘Grep-like’ retrieval, essentially, a powerful search akin to the developer tool ripgrep, and demonstrates that, with careful optimisation, it can outperform state-of-the-art approaches. This research introduces GrepRAG, a framework achieving up to 15.58 percent relative improvement in code completion accuracy, suggesting that lightweight, index-free retrieval deserves renewed attention in the pursuit of efficient and effective code assistance.

Naive GrepRAG delivers competitive code completion via lexical retrieval and generation

Scientists have demonstrated a significant advancement in repository-level code completion for large language models (LLMs) by revisiting a fundamental approach: simple, index-free lexical retrieval. The team achieved performance comparable to sophisticated graph-based methods using a framework called Naive GrepRAG, where LLMs autonomously generate commands for the ripgrep utility to retrieve relevant code context.
This research addresses the challenge of cross-file dependencies and limited context windows that hinder LLM performance in large codebases, a problem previously tackled with computationally expensive semantic indexing or graph analysis. The study reveals that even this basic implementation effectively retrieves lexically precise code fragments located near the completion site, highlighting the potential of lightweight search utilities.

Researchers systematically investigated intent-aware lexical retrieval through extensive empirical analysis, identifying key limitations such as sensitivity to ambiguous keywords and context fragmentation caused by truncation boundaries. To overcome these issues, the team proposed GrepRAG, which augments lexical retrieval with a post-processing pipeline incorporating identifier-weighted re-ranking and structure-aware deduplication.

Extensive evaluation on the CrossCodeEval and RepoEval-Updated datasets demonstrates that GrepRAG consistently outperforms state-of-the-art methods. Specifically, on CrossCodeEval, GrepRAG achieved a 7.04, 15.58 percent relative improvement in code exact match (EM) over the best baseline. This breakthrough establishes that a simple, index-free approach, when optimized, can effectively support repository-level code completion without the substantial computational overhead associated with more complex retrieval mechanisms.

The work opens possibilities for faster and more practical code completion tools, particularly in dynamic software repositories where maintaining up-to-date indexes is challenging. By mirroring common developer workflows that rely on lightweight search utilities like ripgrep, GrepRAG offers a promising solution for enhancing development efficiency and user experience in large-scale software projects. This research proves that intelligent post-processing can significantly enhance the effectiveness of basic lexical retrieval for code completion tasks.

Evaluating Lexical Retrieval and Addressing its Limitations in Code Completion requires careful consideration of context

Scientists developed Naive GrepRAG, a baseline framework where large language models autonomously generate ripgrep commands to retrieve relevant code context. This approach establishes a performance benchmark for repository-level code completion using lightweight, index-free lexical retrieval. The team evaluated Naive GrepRAG against sophisticated graph-based baselines, finding comparable performance despite its simplicity.

Further investigation revealed that the effectiveness of Naive GrepRAG stems from retrieving lexically precise code fragments located spatially close to the completion site. Researchers identified key limitations of lexical retrieval, specifically its sensitivity to noisy matches arising from high-frequency ambiguous keywords.

They also noted context fragmentation caused by rigid truncation boundaries during retrieval. To address these issues, the study pioneered GrepRAG, augmenting lexical retrieval with a lightweight post-processing pipeline. This pipeline features identifier-weighted re-ranking and structure-aware deduplication to refine retrieved code snippets.

Extensive evaluation was conducted on the CrossCodeEval and RepoEval-Updated datasets to assess GrepRAG’s performance. Experiments demonstrated that GrepRAG consistently outperforms state-of-the-art methods, achieving a 7.04-15.58 percent relative improvement in code exact match over the best baseline on CrossCodeEval.

The team measured performance using code exact match, a metric quantifying the precision of generated code completions. The research harnessed the huggingface_diffusers repository, containing approximately 100,000 lines of Python code, for performance analysis. Graph-Coder required 91 seconds to build a graph index and 7 seconds for retrieval on this dataset, highlighting the computational overhead of complex indexing methods.

Developers typically expect latency below 0.5 seconds, with delays exceeding 2 seconds considered unacceptable, motivating the development of faster, index-free approaches like GrepRAG. The study’s methodology enables a re-evaluation of simple lexical retrieval techniques in the context of advanced code completion systems.

GrepRAG demonstrates faster retrieval and improved code matching performance compared to traditional methods

Scientists achieved a 7.04-15.58 percent relative improvement in code exact match (EM) on the CrossCodeEval benchmark using a new framework called GrepRAG, surpassing state-of-the-art methods. Experiments revealed that Naive GrepRAG, a baseline utilising ripgrep for context retrieval, attained performance comparable to sophisticated graph-based baselines despite its simplicity.

The team measured retrieval latency on the RepoEval_Updated dataset, finding that VanillaRAG and GraphCoder required 0.036-7.067 seconds depending on repository size, with larger repositories like FloatingPoint exceeding 50 seconds for a single retrieval. Data shows that ripgrep-based retrieval within the Naive GrepRAG framework required approximately 0.40 seconds for the diffusers repository, significantly less than the 3-7 seconds consumed by baseline methods.

Researchers systematically investigated lightweight, index-free, intent-aware lexical retrieval, demonstrating its effectiveness in retrieving lexically precise code fragments spatially closer to the completion site. Measurements confirm that Naive GrepRAG’s success stems from identifying and retrieving these relevant code segments efficiently.

Analysis of the RepoEval_Updated dataset, comprising repositories ranging from 3.2K to 753.9K lines of code, indicated that existing retrieval methods incur substantial latency when applied to large-scale projects. Specifically, the team recorded retrieval latencies exceeding 2 seconds for repositories like AdaLoRA (97.265 seconds) and FloatingPoint (53.822 seconds) using VanillaRAG.

The study identified keyword ambiguity and context fragmentation as key limitations of Naive GrepRAG, prompting the development of GrepRAG, which incorporates identifier-weighted re-ranking and structure-aware deduplication. Tests prove that GrepRAG addresses these issues, improving completion performance while maintaining efficient retrieval.

The research delineated four retrieval pattern types: class-name, method-name, variable-name, and other, providing insights into the success and failures of retrieval-augmented code completion. The breakthrough delivers a post-processing pipeline that tackles keyword ambiguity, context redundancy, and fragmentation, achieving state-of-the-art performance across multiple benchmarks.

GrepRAG enhances code completion via lexical retrieval and refined re-ranking, improving both speed and accuracy

Researchers have developed GrepRAG, a novel framework for repository-level code completion that leverages lightweight, index-free lexical retrieval. This approach addresses the challenges posed by cross-file dependencies and limited context windows in large language models (LLMs) when applied to extensive codebases.

Naive GrepRAG, a baseline implementation, demonstrated performance comparable to more complex graph-based retrieval methods, suggesting the effectiveness of simple lexical search in identifying relevant code fragments. Further refinement led to the creation of GrepRAG, which incorporates identifier-weighted re-ranking and structure-aware deduplication to mitigate limitations of basic lexical retrieval, such as sensitivity to ambiguous keywords and context fragmentation.

Evaluation on established benchmarks, CrossCodeEval and RepoEval-Updated, revealed that GrepRAG consistently outperformed state-of-the-art methods, achieving relative improvements of 7.04-15.58 percent in code exact match. The authors acknowledge a potential limitation stemming from possible data contamination, where portions of the evaluation benchmarks may have been present in the LLMs’ pre-training data.

However, they argue that the observed performance gains are likely attributable to improved context retrieval, given the consistent outperformance relative to baseline methods using identical backbone models. Future work could explore the application of knowledge distillation techniques, as demonstrated by the efficacy of a distilled 0.6B parameter model, to further enhance efficiency and scalability.