SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot Reinforcement Learning

Dimensions overall score 7.0

• SOLE-R1 enables zero-shot online reinforcement learning from random initialization for previously unseen manipulation tasks without ground-truth rewards, success indicators, demonstrations, or task-specific tuning.result 90%
  Evidencepartial

  SOLE-R1 enables zero-shot online RL from random initialization: robots learn previously unseen manipulation tasks without ground-truth rewards, success indicators, demonstrations, or task-specific tuning.

  Implicationpartial

  Explicitly stated in the abstract as a key result of the work.

  Verificationpartial

  partial

• SOLE-R1 substantially outperforms strong vision-language rewarders, including GPT-5 and Gemini-3-Pro, on manipulation tasks.result 85%
  Evidencepartial

  SOLE-R1 substantially outperforms strong vision-language rewarders, including GPT-5 and Gemini-3-Pro

  Implicationpartial

  Directly stated in the abstract and supported by a results figure (Figure 3) showing success rate comparisons.

  Verificationpartial

  partial

• SOLE-R1 is trained on a dataset of over one million chain-of-thought reasoning examples generated from more than 40,000 real-world and simulated videos.method 90%
  Evidencepartial

  We generate over one million CoT reasoning examples from more than 40,000 real-world and simulated videos.

  Implicationpartial

  Specific numeric data is provided in the paper text.

  Verificationpartial

  partial

• SOLE-R1's training uses a two-stage hybrid framework combining supervised fine-tuning (SFT) and reinforcement learning from verifiable rewards (RLVR).method 90%
  Evidencepartial

  To train SOLE-R1, we propose a two-stage hybrid recipe: SFT teaches high-quality CoT reasoning, while RLVR directly emphasizes accurate progress prediction

  Implicationpartial

  Explicitly described as the core training methodology in Section 4.

  Verificationpartial

  partial

• SOLE-R1 exhibits markedly greater robustness to reward hacking compared to other vision-language models.result 85%
  Evidencepartial

  while exhibiting markedly greater robustness to reward hacking.

  Implicationpartial

  Directly stated in the abstract as a comparative advantage.

  Verificationpartial

  partial

• SOLE-R1 succeeds on 24 unseen tasks across simulation and real-robot settings.result 85%
  Evidencepartial

  SOLE-R1 succeeds on 24 unseen tasks

  Implicationpartial

  Specific numeric claim made in the abstract.

  Verificationpartial

  partial

• The model performs per-timestep spatiotemporal chain-of-thought reasoning to produce dense estimates of task progress that can be used directly as rewards.technical 90%
  Evidencepartial

  SOLE-R1 performs per-timestep spatiotemporal chain-of-thought (CoT) reasoning and produces dense estimates of task progress that can be used directly as rewards.

  Implicationpartial

  Core technical capability is explicitly defined in the abstract and Section 2.

  Verificationpartial

  partial

• Supervised fine-tuning alone provides a weak learning signal for reward/progress prediction because the scalar answer occupies only a small fraction of response tokens.Source prooflimitation 80%
  Evidenceverified

  it provides a relatively weak learning signal for reward/progress prediction, since the scalar in is a small part of the response.

  Implicationpartial

  Directly stated as a limitation of the SFT stage, justifying the need for the RLVR stage.

  Verificationverified

  verified

Competitive landscape

A video-language reasoning model that acts as the sole reward signal for robots, enabling them to learn new manipulation tasks without human supervision or ground truth rewards.