Quantization Dominates Rank Reduction for KV-Cache Compression

Dimensions overall score 3.0

• Quantization is a more effective method than rank reduction for compressing KV caches in transformer inference, showing significant performance gains. At matched storage budgets across five models (124M-14B, MHA and GQA), we find that quantization consistently outperforms rank reduction by 4-364 PPL depending on model and compression level.problem 35%
  Evidencepartial

  Quantization is a more effective method than rank reduction for compressing KV caches in transformer inference, showing significant performance gains. At matched storage budgets across five models (124M-14B, MHA and GQA), we find that quantization consistently outperforms rank reduction by 4-364 PPL depending on model and compression level.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• We compare two strategies for compressing the KV cache in transformer inference: rank reduction (discard dimensions) and quantization (keep all dimensions, reduce precision). At matched storage budgets across five models (124M-14B, MHA and GQA), we find that quantization consistently outperforms rank reduction by 4-364 PPL depending on model and compression level.method 35%
  Evidencepartial

  We compare two strategies for compressing the KV cache in transformer inference: rank reduction (discard dimensions) and quantization (keep all dimensions, reduce precision). At matched storage budgets across five models (124M-14B, MHA and GQA), we find that quantization consistently outperforms rank reduction by 4-364 PPL depending on model and compression level.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• ScienceToStartup currently rates this 3.0/10 on the public viability pass. We formalize this via a perturbation result showing projection damage exceeds quantization damage by 3 x 2^(2b) per direction under the softmax Fisher metric.result 35%
  Evidencepartial

  ScienceToStartup currently rates this 3.0/10 on the public viability pass. We formalize this via a perturbation result showing projection damage exceeds quantization damage by 3 x 2^(2b) per direction under the softmax Fisher metric.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• LLM Inference Optimization moved forward this cycle; last verified April 2026. Public score 3.0/10.why now 35%
  Evidencepartial

  LLM Inference Optimization moved forward this cycle; last verified April 2026. Public score 3.0/10.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

Competitive landscape

Quantization is a more effective method than rank reduction for compressing KV caches in transformer inference, showing significant performance gains.