IMSE: Intrinsic Mixture of Spectral Experts Fine-tuning for Test-Time Adaptation

Dimensions overall score 8.0

• IMSE achieves state-of-the-art performance on various distribution-shift benchmarks under the TTA setting.result 90%
  Evidencepartial

  Consequently, our method achieves state-of-the-art performance on various distribution-shift benchmarks under the TTA setting.

  Implicationpartial

  Directly stated in abstract with clear performance claim

  Verificationpartial

  partial

• IMSE improves accuracy by 3.4 percentage points in CTTA and 2.4 percentage points in Gradual CTTA compared to previous methods.result 90%
  Evidencepartial

  In CTTA and Gradual CTTA, it further improves accuracy by 3.4 percentage points (pp) and 2.4 pp, respectively

  Implicationpartial

  Direct numeric evidence provided in abstract

  Verificationpartial

  partial

• IMSE requires 385 times fewer trainable parameters than previous methods.technical 90%
  Evidencepartial

  while requiring 385 times fewer trainable parameters.

  Implicationpartial

  Direct numeric evidence provided in abstract

  Verificationpartial

  partial

• Entropy minimization in TTA often induces feature collapse, causing models to rely on domain-specific features rather than class-discriminative features.limitation 85%
  Evidencepartial

  We further identify a key limitation of entropy minimization in TTA: it often induces feature collapse, causing the model to rely on domain-specific features rather than class-discriminative features.

  Implicationpartial

  Directly stated as a key limitation in abstract

  Verificationpartial

  partial

• IMSE adapts only the singular values while keeping singular vectors fixed during fine-tuning.method 90%
  Evidencepartial

  We decompose each linear layer via singular value decomposition (SVD) and adapt only the singular values, while keeping the singular vectors fixed.

  Implicationpartial

  Directly stated method description in abstract

  Verificationpartial

  partial

• IMSE introduces a diversity maximization loss based on expert-input alignment to encourage diverse utilization of spectral experts.method 90%
  Evidencepartial

  To address this, we propose a diversity maximization loss based on expert-input alignment, which encourages diverse utilization of spectral experts during adaptation.

  Implicationpartial

  Directly stated method component in abstract

  Verificationpartial

  partial

• IMSE introduces Domain-Aware Spectral Code Retrieval to detect domain shifts and retrieve adapted singular values for rapid adaptation.method 90%
  Evidencepartial

  We introduce Domain-Aware Spectral Code Retrieval, which estimates input distributions to detect domain shifts, and retrieves adapted singular values for rapid adaptation.

  Implicationpartial

  Directly stated method component in abstract

  Verificationpartial

  partial

• Fully leveraging rich representations of large pretrained models with minimal parameter updates remains underexplored in TTA.technical 80%
  Evidencepartial

  However, fully leveraging the rich representations of large pretrained models with minimal parameter updates remains underexplored.

  Implicationpartial

  Directly stated as motivation in abstract, though somewhat subjective

  Verificationpartial

  partial

Competitive landscape

IMSE leverages spectral experts in Vision Transformers for test-time adaptation, offering state-of-the-art performance with minimal parameter updates and domain-aware adaptation.