Frequency Matters: Fast Model-Agnostic Data Curation for Pruning and Quantization

Dimensions overall score 8.0

GitHub Code Pulse

No public code linked for this paper yet.

• ZipCal is a model-agnostic data curation strategy that maximizes lexical diversity based on Zipfian power laws.method 100%
• ZipCal consistently outperforms standard uniform random sampling across various pruning benchmarks.result 90%
• ZipCal performs on par with a state-of-the-art method relying on model perplexity in terms of downstream performance.result 90%
• ZipCal is approximately 240 times faster than methods relying on model perplexity.technical 90%
• Methods relying on model perplexity become prohibitively expensive at large-scale models and datasets.limitation 90%
• The proposed method addresses the challenge of identifying high-performance calibration sets for both pruning and quantization.method 100%
• The choice of calibration data is critical for preserving model capabilities both intra- and inter-tasks.technical 90%

Competitive landscape

Competitor map is still being generated for this paper. Enable generation or check back soon.

Startup potential card