Llms Achieve 56% Compression with Adaptive Pruning, Maintaining Factual Knowledge

The escalating size of Large Language Models (LLMs) presents a significant challenge for computational efficiency, prompting researchers to explore methods for model compression. Sai Varun Kodathala from Sports Vision, Inc., and Rakesh Vunnam from Vizworld Inc., alongside their colleagues, address this issue with a novel approach to post-training pruning. Their work introduces agent-guided pruning, a technique where an LLM acts as an intelligent agent to dynamically determine which layers of a model should be pruned, preserving crucial knowledge during the process. This research is significant because it not only achieves substantial compression , reaching approximately 45% sparsity , but also demonstrably mitigates the factual knowledge degradation commonly associated with pruned LLMs, showing improvements in accuracy and perplexity compared to existing structured pruning methods. Through a self-correcting mechanism and framework-agnostic design, the team demonstrates that LLMs can effectively guide the compression of other LLMs without the need for resource-intensive retraining.

Achieving high sparsity through layer-wise weight reconstruction or activation-aware magnitude pruning is a common goal, yet current methods often depend on uniform or hand-crafted heuristics to determine per-layer sparsity ratios. This work introduces agent-guided pruning, a novel approach where a foundation model functions as an adaptive pruning agent to intelligently select layers for pruning at each iteration, thereby preserving critical knowledge pathways. The method constructs layer-wise sensitivity profiles by combining Wanda-inspired weight-activation analysis with a knowledge-aware probing mechanism. This allows for a more nuanced and effective pruning strategy than previously available, mitigating the risk of catastrophic knowledge loss during model compression.

LLM Compression via Agent-Guided Adaptive Pruning

Researchers have developed a new adaptive pruning framework to compress large language models (LLMs) without requiring retraining. The methodology centres on iteratively removing unimportant connections, a process known as pruning, guided by a large language model agent. This agent assesses the impact of each pruning decision using validation metrics and gradient importance scores, which are then normalised as z-scores to allow comparison across different models. The system employs a self-reflection mechanism, enabling the LLM agent to learn from previous pruning iterations and refine its strategy accordingly.

A checkpoint rollback system is also incorporated; if the model’s performance, measured by perplexity degradation, falls below a pre-defined threshold, the model reverts to a previous, better-performing state. This ensures model quality is maintained throughout the compression process, and the framework operates in a model-agnostic manner, meaning it is not limited to specific LLM architectures. Experiments were conducted on Qwen3 models, with parameter counts of 4 billion and 8 billion, achieving approximately 45% sparsity. Results demonstrate significant improvements compared to standard structured pruning techniques.

Specifically, the new framework achieved a 56% relative improvement in MMLU accuracy, a 19-fold increase in factual knowledge retention on the FreebaseQA benchmark, and a 69% reduction in perplexity degradation. Across 21 to 40 iterations, the framework required only 2-4 checkpoint rollbacks, indicating effective self-correction. This suggests that foundation models can successfully guide the compression of other foundation models, offering a promising approach to deploying LLMs in resource-constrained environments and addressing the issue of catastrophic factual knowledge loss often seen in pruned models.

Agent-Guided Pruning Boosts LLM Efficiency and Accuracy

Scientists have achieved a breakthrough in large language model (LLM) compression, demonstrating a novel agent-guided pruning technique that significantly reduces computational costs without sacrificing performance. The research team successfully pruned Qwen3 models, both 4 billion and 8 billion parameters in size, to approximately 45% sparsity, employing an LLM agent to intelligently select layers for pruning and preserve critical knowledge pathways. Experiments revealed a substantial 56% relative improvement in MMLU accuracy compared to established structured pruning baselines, showcasing the method’s ability to maintain general language understanding capabilities. Data shows the framework excels in preserving factual knowledge, achieving a remarkable 19-fold improvement in factual knowledge retention on the FreebaseQA benchmark.

This addresses a critical limitation of previous pruning methods, which often suffer severe degradation in factual recall, even with minimal sparsity. Measurements confirm a 69% lower perplexity degradation, indicating the pruned models maintain a higher level of fluency and coherence in generated text compared to existing techniques. The team constructed layer-wise sensitivity profiles by combining weight-activation metrics with gradient importance scores, normalized using z-scores to enable model-agnostic comparison across different layer types. The core of this advancement lies in the use of a self-reflective LLM agent, which learns from previous pruning outcomes and iteratively refines its strategy.

This agent intelligently assesses which layers can be safely pruned while preserving essential knowledge, guided by a checkpoint rollback mechanism that reverts the model when performance dips below a defined threshold. Tests prove the system’s robustness, exhibiting effective self-correction with only 2-4 rollbacks across 21 to 40 iterations, demonstrating the agent’s capacity to autonomously optimize the pruning process. Notably, this framework requires no retraining of the LLM, operates independently of the underlying model architecture, and achieves these results with a remarkably low rollback rate of 9.5-10%. The breakthrough delivers a new paradigm for automated neural architecture optimization, establishing that foundation models can effectively guide the compression of other foundation models, opening avenues for more efficient and accessible artificial intelligence. This work establishes a foundation for further research into adaptive pruning strategies and their potential to unlock the full capabilities of large language models.