WriteSAE: Sparse Autoencoders for Recurrent State

Dimensions overall score 7.0

• WriteSAE: Sparse autoencoders that decompose and edit the matrix cache write of recurrent language models, enabling precise behavioral modification. Existing SAEs read residual streams, but Gated DeltaNet, Mamba-2, and RWKV-7 write to a $d_k \times d_v$ cache through rank-1 updates $k_t v_t^\top$ that no vector atom can replace.problem 35%
  Evidencepartial

  WriteSAE: Sparse autoencoders that decompose and edit the matrix cache write of recurrent language models, enabling precise behavioral modification. Existing SAEs read residual streams, but Gated DeltaNet, Mamba-2, and RWKV-7 write to a $d_k \times d_v$ cache through rank-1 updates $k_t v_t^\top$ that no vector atom can replace.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• We introduce WriteSAE, the first sparse autoencoder that decomposes and edits the matrix cache write of state-space and hybrid recurrent language models, where residual SAEs cannot reach. Existing SAEs read residual streams, but Gated DeltaNet, Mamba-2, and RWKV-7 write to a $d_k \times d_v$ cache through rank-1 updates $k_t v_t^\top$ that no vector atom can replace.method 35%
  Evidencepartial

  We introduce WriteSAE, the first sparse autoencoder that decomposes and edits the matrix cache write of state-space and hybrid recurrent language models, where residual SAEs cannot reach. Existing SAEs read residual streams, but Gated DeltaNet, Mamba-2, and RWKV-7 write to a $d_k \times d_v$ cache through rank-1 updates $k_t v_t^\top$ that no vector atom can replace.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• ScienceToStartup currently rates this 7.0/10 on the public viability pass. Sustained three-position installs at $3\times$ lift midrank target-in-continuation from 33.3% to 100% under greedy decoding, the first behavioral install at the matrix-recurrent write site. A public repository is linked, so build verification can inspect implementation evidence instead of treating the paper as PDF-only.result 35%
  Evidencepartial

  ScienceToStartup currently rates this 7.0/10 on the public viability pass. Sustained three-position installs at $3\times$ lift midrank target-in-continuation from 33.3% to 100% under greedy decoding, the first behavioral install at the matrix-recurrent write site. A public repository is linked, so build verification can inspect implementation evidence instead of treating the paper as PDF-only.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• LLM Internals moved forward this cycle; last verified May 2026. Public score 7.0/10. Implementation evidence is present through a linked repository.why now 35%
  Evidencepartial

  LLM Internals moved forward this cycle; last verified May 2026. Public score 7.0/10. Implementation evidence is present through a linked repository.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

Competitive landscape

WriteSAE: Sparse autoencoders that decompose and edit the matrix cache write of recurrent language models, enabling precise behavioral modification.