Dimensions overall score 7.0

• The HyperP framework enables the transfer of a single base learning rate tuned at the smallest scale across all compute budgets, achieving a 1.58x compute efficiency improvement over a strong Muon baseline at 6e21 FLOPs.result 90%
  Evidencepartial

  A single base learning rate tuned at the smallest scale transfers across all compute budgets under HyperP, yielding $1.58\times$ compute efficiency over a strong Muon baseline at $6\times10^{21}$ FLOPs.

  Implicationpartial

  Directly stated in the abstract with a specific numeric result.

  Verificationpartial

  partial

• Hypersphere optimization with HyperP provides transferable stability, keeping instability indicators like Z-values, output RMS, and activation outliers bounded and non-increasing when scaling training FLOPs.result 90%
  Evidencepartial

  HyperP delivers transferable stability: all monitored instability indicators, including $Z$-values, output RMS, and activation outliers, remain bounded and non-increasing under training FLOPs scaling.

  Implicationpartial

  Explicitly claimed in the abstract with clear metrics listed.

  Verificationpartial

  partial

• Weight decay is a first-order no-op (has no first-order effect) on the Frobenius sphere under the hypersphere optimization framework.Source prooftechnical 85%
  Evidenceverified

  We prove that weight decay is a first-order no-op on the Frobenius sphere

  Implicationpartial

  Stated as a proven theoretical result in the abstract.

  Verificationverified

  verified

• The optimal learning rate under HyperP follows the same data-scaling power law with an exponent of 0.32, previously observed for AdamW.result 85%
  Evidencepartial

  find that the optimal learning rate follows the same data-scaling power law with the "magic exponent" 0.32 previously observed for AdamW.

  Implicationpartial

  Directly stated in the abstract with a specific numeric exponent.

  Verificationpartial

  partial

• The SqrtGate MoE gating mechanism, derived from the hypersphere constraint, preserves output RMS across MoE granularities for improved granularity scaling.method 85%
  Evidencepartial

  We also propose SqrtGate, an MoE gating mechanism derived from the hypersphere constraint that preserves output RMS across MoE granularities for improved granularity scaling

  Implicationpartial

  Explicitly proposed in the abstract with a stated purpose.

  Verificationpartial

  partial

• Hypersphere optimization enables the use of substantially larger auxiliary load-balancing weights in MoE models, yielding both strong performance and good expert balance.result 80%
  Evidencepartial

  show that hypersphere optimization enables substantially larger auxiliary load-balancing weights, yielding both strong performance and good expert balance.

  Implicationpartial

  Directly stated in the abstract as a benefit of the method.

  Verificationpartial

  partial

• Existing hyperparameter transfer laws are mainly developed for first-order optimizers and do not structurally prevent training instability at scale.limitation 90%
  Evidencepartial

  Existing hyperparameter transfer laws are mainly developed for first-order optimizers, and they do not structurally prevent training instability at scale.

  Implicationpartial

  Directly stated as a limitation of prior work in the abstract.

  Verificationpartial

  partial

• Depth-μP remains a necessary component even under the Frobenius-sphere constraint with the Muon optimizer.technical 80%
  Evidencepartial

  show that Depth-$\mu$P remains necessary

  Implicationpartial

  Stated as a finding in the abstract, though the exact necessity is a conclusion.

  Verificationpartial

  partial

Competitive landscape

A new hypersphere parameterization framework that enables stable and efficient scaling of large language models with transferable learning rates.