$f$-Trajectory Balance: A Loss Family for Tuning GFlowNets, Generative Models, and LLMs with Off- and On-Policy Data

Dimensions overall score 5.0

• A family of surrogate loss functions for training generative models, including GFlowNets, LLMs, and variational inference, that can utilize both off- and on-policy data. This loss has the property that when evaluated \emph{on-policy} its gradients correspond to those of the KL divergence, while \emph{off-policy} it remains a valid loss with the same global minimizer.problem 35%
  Evidencepartial

  A family of surrogate loss functions for training generative models, including GFlowNets, LLMs, and variational inference, that can utilize both off- and on-policy data. This loss has the property that when evaluated \emph{on-policy} its gradients correspond to those of the KL divergence, while \emph{off-policy} it remains a valid loss with the same global minimizer.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• In GFlowNets and variational inference, it has been shown that the mean square error between target and model log probabilities is an effective, low variance, surrogate loss for training generative models. This loss has the property that when evaluated \emph{on-policy} its gradients correspond to those of the KL divergence, while \emph{off-policy} it remains a valid loss with the same global minimizer.method 35%
  Evidencepartial

  In GFlowNets and variational inference, it has been shown that the mean square error between target and model log probabilities is an effective, low variance, surrogate loss for training generative models. This loss has the property that when evaluated \emph{on-policy} its gradients correspond to those of the KL divergence, while \emph{off-policy} it remains a valid loss with the same global minimizer.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• ScienceToStartup currently rates this 5.0/10 on the public viability pass. In this work, we demonstrate that this construction can be extended to the whole family of $f$-divergences, leading to a family of losses whose on-policy gradients are that of the corresponding $f$-divergence, but retain the same global minimizer off-policy. Code availability is flagged in the production record; the public repository link still needs proof alignment.result 35%
  Evidencepartial

  ScienceToStartup currently rates this 5.0/10 on the public viability pass. In this work, we demonstrate that this construction can be extended to the whole family of $f$-divergences, leading to a family of losses whose on-policy gradients are that of the corresponding $f$-divergence, but retain the same global minimizer off-policy. Code availability is flagged in the production record; the public repository link still needs proof alignment.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• Generative Model Training moved forward this cycle; last verified May 2026. Public score 5.0/10. Production flags indicate code availability.why now 35%
  Evidencepartial

  Generative Model Training moved forward this cycle; last verified May 2026. Public score 5.0/10. Production flags indicate code availability.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

Competitive landscape

A family of surrogate loss functions for training generative models, including GFlowNets, LLMs, and variational inference, that can utilize both off- and on-policy data.