HandX: Scaling Bimanual Motion and Interaction Generation

Dimensions overall score 7.0

• HandX introduces a new motion-capture dataset of 54.2 hours (5.9M frames) targeting underrepresented bimanual interactions with detailed finger dynamics.method 95%
  Evidencepartial

  We consolidate and filter existing datasets for quality, and collect a new motion-capture dataset targeting underrepresented bimanual interactions with detailed finger dynamics.

  Implicationpartial

  Explicitly stated in the abstract and supported by the dataset scale table (Table 1).

  Verificationpartial

  partial

• HandX's annotation strategy uses a decoupled approach that extracts motion features and leverages large language models to produce fine-grained, semantically rich descriptions.method 90%
  Evidencepartial

  For scalable annotation, we introduce a decoupled strategy that extracts representative motion features, e.g., contact events and finger flexion, and then leverages reasoning from large language models to produce fine-grained, semantically rich descriptions aligned with these features.

  Implicationpartial

  Directly stated in the abstract as a core methodological contribution.

  Verificationpartial

  partial

• Larger models trained on larger, higher-quality datasets produce more semantically coherent bimanual motion, showing a clear scaling trend.result 85%
  Evidencepartial

  We further observe clear scaling trends: larger models trained on larger, higher-quality datasets produce more semantically coherent bimanual motion.

  Implicationpartial

  Explicitly stated in the abstract and supported by a figure showing a log-linear relationship between R-precision and FLOPS.

  Verificationpartial

  partial

• HandX's dataset provides significantly more fine-grained text annotations (485.7K) compared to existing datasets like HoloAssist (1.8K).result 90%
  Evidencepartial

  HandX (Ours) 54.2 5.9 fine-grained 485.7

  Implicationpartial

  Direct numeric comparison provided in Table 1, showing HandX's 'fine-grained' annotations at 485.7K vs. others with 'coarse' annotations at much lower counts.

  Verificationpartial

  partial

• A diffusion model architecture with separate text encoders for left-hand, right-hand, and bimanual interaction prompts improves performance over simple concatenation.technical 80%
  Evidencepartial

  We observe that simply concatenating the three types of prompts degrades performance,e.g., the generated motion may assign right-hand movements to the left hand. To address this issue, we encode three types of prompts separately and add a learnable CLS token to each...

  Implicationpartial

  Described in the model architecture section, stating that simple concatenation degrades performance and the proposed separate encoding addresses this.

  Verificationpartial

  partial

• Existing datasets like CLUTCH and InterHand2.6M lack the precision, diversity, and rich inter-hand contact needed for learning fine-grained bimanual motion from text.limitation 85%
  Evidencepartial

  Overall, existing datasets still lack the precision, diversity, and rich inter-hand contact needed for learning fine-grained bimanual motion from text.

  Implicationpartial

  Directly stated as a limitation of prior work that motivates the HandX dataset.

  Verificationpartial

  partial

• The proposed benchmark models (diffusion and autoregressive) support versatile conditioning modes for bimanual motion generation.method 80%
  Evidencepartial

  Building on the resulting data and annotations, we benchmark diffusion and autoregressive models with versatile conditioning modes.

  Implicationpartial

  Stated in the abstract and supported by the model architecture description showing different conditioning inputs.

  Verificationpartial

  partial

• HandX introduces new hand-focused metrics (contact precision, recall, F1) to evaluate hand contact accuracy in generated motions.method 85%
  Evidencepartial

  We propose contact precision (Cprec), contact recall (Crec), and F1 score (CF1) to evaluate hand contact accuracy.

  Implicationpartial

  Explicitly described in the evaluation section as newly proposed metrics.

  Verificationpartial

  partial

Competitive landscape

A unified foundation for generating realistic bimanual hand motion, including data, annotation, and evaluation, with a focus on fine-grained finger dynamics and inter-hand coordination.