HeteroCache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inference

HeteroCache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inference explores HeteroCache offers a high-performance, training-free dynamic compression framework to optimize LLM inference in long-context tasks.. Commercial viability score: 7/10 in LLM Optimization.