Seen2Scene: Completing Realistic 3D Scenes with Visibility-Guided Flow

Dimensions overall score 7.0

• Seen2Scene is the first flow matching-based approach that trains directly on incomplete, real-world 3D scans for scene completion and generation.method 95%
  Evidencepartial

  We present Seen2Scene, the first flow matching-based approach that trains directly on incomplete, real-world 3D scans for scene completion and generation.

  Implicationpartial

  Explicitly stated in the abstract as a primary contribution of the paper.

  Verificationpartial

  partial

• Seen2Scene's visibility-guided flow matching, which masks unknown regions, enables effective learning from real-world, partial observations.method 90%
  Evidencepartial

  our approach introduces visibility-guided flow matching, which explicitly masks out unknown regions in real scans, enabling effective learning from real-world, partial observations.

  Implicationpartial

  Directly stated in the abstract as the core methodological innovation, with ablation study (Fig. 7) showing its necessity.

  Verificationpartial

  partial

• Seen2Scene outperforms baseline methods (SG-NN, NKSR) in 3D scan completion accuracy.result 85%
  Evidencepartial

  Our model without bounding boxes achieves comparable performance to its bounding box-conditioned counterpart, and even yields better overall structural correctness (CD).

  Implicationpartial

  Supported by quantitative results in Tab. 1 (referenced in text) showing better metrics for Seen2Scene.

  Verificationpartial

  partial

• Seen2Scene produces more geometrically detailed and semantically coherent scenes in layout-conditioned generation compared to BlockFusion, LT3SD, and WorldGrow.result 80%
  Evidencepartial

  Our method produces more geometrically detailed and semantically coherent scenes compared to BlockFusion [61], LT3SD [42], and WorldGrow [34].

  Implicationpartial

  Qualitative claim made in caption of Fig. 5 and supported by quantitative metrics in a table (DINOv2-FID, U3D-FPD, VLM Score).

  Verificationpartial

  partial

• Training without masked flow matching causes the model to generate holes in scenes, following the pattern of incomplete training scans.technical 90%
  Evidencepartial

  Without two stages of masked training, the model tends to generate holes (under the table) following the training incomplete scans.

  Implicationpartial

  Directly stated in the analysis of the ablation study (Fig. 7) with a specific example.

  Verificationpartial

  partial

• Using CLIP embeddings instead of one-hot semantic labels improves generalization to diverse object layouts.technical 75%
  Evidencepartial

  Open Vocabulary Semantic Encodings.We ablate the effect of using CLIP embedding [46] instead of one-hot semantic labels which has a fixed number of categories

  Implicationpartial

  Implied by the ablation study in Tab. 4 and Fig. 8, which discusses the challenge of category mapping with fixed labels.

  Verificationpartial

  partial

• Seen2Scene can be adapted for text-to-scene generation by first generating a 3D object layout from the text description.method 85%
  Evidencepartial

  Text to scene.Given a text description, Seen2Scene generates a 3D object layout first via an

  Implicationpartial

  Explicitly stated and demonstrated with qualitative examples in Fig. 6.

  Verificationpartial

  partial

• The model's training on datasets with limited object categories (like 3D-FRONT) reduces its generalization ability to diverse, cluttered real-world layouts.limitation 70%
  Evidencepartial

  training only on 3D-FRONT [20], or with limited categories, or without label synonym augmentation, reduces the model’s generalization ability to diverse object layouts, like 'clothes on an office chair.'

  Implicationpartial

  Stated as a finding from an ablation study, though the specific performance degradation is not quantified.

  Verificationpartial

  partial

Competitive landscape

Generate realistic and complete 3D scenes from incomplete real-world scans using visibility-guided flow matching.