LLMs can Compress LLMs: Adaptive Pruning by Agents

Dimensions overall score 8.0

GitHub Code Pulse

No public code linked for this paper yet.

• Agent-guided pruning achieves 56% relative improvement in MMLU accuracy over structured pruning baselines at approximately 45% sparsity.result 90%
• Agent-guided pruning achieves 19x better factual knowledge retention on FreebaseQA compared to structured pruning baselines.result 90%
• Agent-guided pruning achieves 69% lower perplexity degradation compared to structured pruning baselines.result 90%
• The method constructs layer-wise sensitivity profiles by combining Wanda-inspired weight-activation metrics with gradient importance scores, normalized as z-scores for model-agnostic comparison.method 85%
• The framework requires no retraining and operates in a model-agnostic manner.technical 85%
• The method exhibits effective self-correction with only 2-4 rollbacks across 21-40 iterations.result 85%
• Existing pruning methods such as SparseGPT and Wanda rely on uniform or hand-crafted heuristics to determine per-layer sparsity ratios.limitation 80%
• Pruned LLMs suffer from severe factual knowledge degradation, with structured pruning methods experiencing near-total collapse in factual question-answering capabilities.limitation 80%

Competitive landscape

Competitor map is still being generated for this paper. Enable generation or check back soon.

Startup potential card