Gradient Atoms: Unsupervised Discovery, Attribution and Steering of Model Behaviors via Sparse Decomposition of Training Gradients

Dimensions overall score 8.0

• Gradient Atoms decomposes per-document training gradients into sparse components via dictionary learning in a preconditioned eigenspace.method 95%
  Evidencepartial

  We present Gradient Atoms, an unsupervised method that decomposes per-document training gradients into sparse components ("atoms") via dictionary learning in a preconditioned eigenspace.

  Implicationpartial

  Directly stated in abstract as core method description

  Verificationpartial

  partial

• Among 500 discovered atoms, the highest-coherence ones recover interpretable task-type behaviors without any behavioral labels.result 90%
  Evidencepartial

  Among the 500 discovered atoms, the highest-coherence ones recover interpretable task-type behaviors -- refusal, arithmetic, yes/no classification, trivia QA -- without any behavioral labels.

  Implicationpartial

  Directly stated in abstract with specific number of atoms and clear outcome

  Verificationpartial

  partial

• Applying discovered atoms as weight-space perturbations produces large, controllable shifts in model behavior.result 90%
  Evidencepartial

  These atoms double as effective steering vectors: applying them as weight-space perturbations produces large, controllable shifts in model behavior (e.g., bulleted-list generation 33% to 94%; systematic refusal 50% to 0%).

  Implicationpartial

  Directly stated in abstract with specific numeric examples provided

  Verificationpartial

  partial

• The method requires no query-document scoring stage and scales independently of the number of query behaviors of interest.technical 85%
  Evidencepartial

  The method requires no query-document scoring stage, and scales independently of the number of query behaviors of interest.

  Implicationpartial

  Directly stated in abstract as key advantage over existing methods

  Verificationpartial

  partial

• Existing TDA methods are supervised, requiring a query behavior and scoring every training document against it.limitation 90%
  Evidencepartial

  Existing TDA methods are supervised -- they require a query behavior, then score every training document against it -- making them both expensive and unable to surface behaviors the user did not think to ask about.

  Implicationpartial

  Directly stated in abstract as limitation of existing approaches

  Verificationpartial

  partial

• Per-document framing of training data attribution is fundamentally mismatched to how fine-tuning actually works.technical 80%
  Evidencepartial

  We argue that this per-document framing is fundamentally mismatched to how fine-tuning actually works: models often learn broad concepts shared across many examples.

  Implicationpartial

  Directly stated in abstract as core argument, though more conceptual than empirical

  Verificationpartial

  partial

• Steering vectors could have unintended side effects on model performance.limitation 70%
  Evidencepartial

  Steering vectors could have unintended side effects on model performance.

  Implicationpartial

  Stated in analysis caveats section, though not in main paper text

  Verificationpartial

  partial

• The method may not generalize to all model architectures or training regimes.limitation 70%
  Evidencepartial

  Method may not generalize to all model architectures or training regimes.

  Implicationpartial

  Stated in analysis caveats section, though not in main paper text

  Verificationpartial

  partial

Competitive landscape

Gradient Atoms offers an unsupervised method for discovering and steering model behaviors through sparse decomposition of training gradients.