Scene Grounding In the Wild

Dimensions overall score 7.0

• The proposed framework enables globally consistent alignment of partial 3D reconstructions even in the absence of visual overlap by grounding them to a complete reference model.method 95%
  Evidencepartial

  In such cases, existing reconstruction pipelines often produce multiple disconnected partial reconstructions or erroneously merge non-overlapping regions into overlapping geometry. In this work, we propose a framework that grounds each partial reconstruction to a complete reference model of the scene, enabling globally consistent alignment even in the absence of visual overlap.

  Implicationpartial

  This is a core claim stated in the abstract and is the central theme of the paper.

  Verificationpartial

  partial

• The reference models are derived from dense, geospatially accurate pseudo-synthetic renderings from Google Earth Studio, which provide full scene coverage but have a significant appearance domain gap compared to real-world photographs.method 95%
  Evidencepartial

  We obtain reference models from dense, geospatially accurate pseudo-synthetic renderings derived from Google Earth Studio. These renderings provide full scene coverage but differ substantially in appearance from real-world photographs.

  Implicationpartial

  This is explicitly stated in the abstract as the source of the reference models and highlights a key challenge.

  Verificationpartial

  partial

• The alignment approach utilizes semantic features from 3D Gaussian Splatting representations of the reference model and formulates alignment as an inverse feature-based optimization scheme.method 95%
  Evidencepartial

  We represent the reference model using 3D Gaussian Splatting, augmenting each Gaussian with semantic features, and formulate alignment as an inverse feature-based optimization scheme that estimates a global 6DoF pose and scale while keeping the reference model fixed.

  Implicationpartial

  The abstract clearly outlines the technical approach for alignment.

  Verificationpartial

  partial

• The proposed approach consistently improves global alignment when initialized with various classical and learning-based pipelines and mitigates failure modes of state-of-the-art end-to-end models.result 95%
  Evidencepartial

  We demonstrate that our approach consistently improves global alignment when initialized with various classical and learning-based pipelines, while mitigating failure modes of state-of-the-art end-to-end models.

  Implicationpartial

  This is a key result presented in the abstract, summarizing the performance benefits.

  Verificationpartial

  partial

• The WikiEarth dataset registers existing partial 3D reconstructions with pseudo-synthetic reference models.method 95%
  Evidencepartial

  Furthermore, we introduce the WikiEarth dataset, which registers existing partial 3D reconstructions with pseudo-synthetic reference models.

  Implicationpartial

  The abstract explicitly introduces the WikiEarth dataset and its purpose.

  Verificationpartial

  partial

• State-of-the-art feed-forward reconstruction models like DUSt3R, MASt3R, π3, and VGGT frequently collapse non-overlapping partial reconstructions into incorrect geometries on the benchmark, highlighting the need for an external reference model.result 90%
  Evidencepartial

  Our evaluation shows that despite strong performance in other settings, on our benchmark they frequently collapse non-overlapping partial reconstructions into incorrect geometries, highlighting the need for an external reference model and our semantic-based alignment approach.

  Implicationpartial

  The paper explicitly states that these models struggle with the benchmark and require an external reference.

  Verificationpartial

  partial

• VF-NeRF, which aligns scenes through differentiable rendering, significantly underperforms on the 'in-the-wild' dataset due to large color variations and low-quality reference models, resulting in high rotation and translation errors.result 90%
  Evidencepartial

  VF-NeRF [47], effectively aligns scenes through differen-tiable rendering in controlled environments. However, this method struggles on our “in-the-wild” dataset, as our results show it significantly underperforms, with high rotation er-ror (∆Rof 6.48) and translation error (∆Tof 0.38), likely due to large color variations between meta-images and the low-quality reference model.

  Implicationpartial

  The paper provides specific performance metrics for VF-NeRF and explains the reasons for its underperformance.

  Verificationpartial

  partial

• LSeg, while effective for segmentation, lacks the robustness for large-scale scene alignment in the challenging 'in-the-wild' setting and fails to outperform even the initialization (COLMAP).result 90%
  Evidencepartial

  LSeg fails to outperform even the initialization (COLMAP), yielding a translation error of 0.34. This suggests that LSeg, though effective for segmentation, lacks the robust-ness needed for large-scale scene alignment in this chal-lenging setting.

  Implicationpartial

  The paper explicitly states LSeg's limitations and performance relative to the baseline.

  Verificationpartial

  partial

Competitive landscape

A framework for globally consistent 3D scene reconstruction from sparse, in-the-wild imagery by aligning partial reconstructions to semantic features of reference models.