Omni123: Exploring 3D Native Foundation Models with Limited 3D Data by Unifying Text to 2D and 3D Generation

Dimensions overall score 7.0

• Omni123 is a 3D-native foundation model that unifies text-to-2D and text-to-3D generation within a single autoregressive framework.method 95%
  Evidencepartial

  We present Omni123, a 3D-native foundation model that unifies text-to-2D and text-to-3D generation within a single autoregressive framework.

  Implicationpartial

  Explicitly stated in the abstract as the core contribution of the paper.

  Verificationpartial

  partial

• The model leverages abundant 2D data as a geometric prior to improve 3D representations, addressing the scarcity of high-quality 3D assets.method 90%
  Evidencepartial

  By representing text, images, and 3D as discrete tokens in a shared sequence space, the model leverages abundant 2D data as a geometric prior to improve 3D representations.

  Implicationpartial

  Directly stated as the key insight and method to overcome the data limitation problem.

  Verificationpartial

  partial

• The training paradigm coordinates diverse cross-modal tasks without requiring fully aligned text-image-3D triplets.method 85%
  Evidencepartial

  We introduce an interleaved X-to-X training paradigm that coordinates diverse cross-modal tasks over heterogeneous paired datasets without requiring fully aligned text-image-3D triplets.

  Implicationpartial

  Explicitly stated as a feature of the introduced training method.

  Verificationpartial

  partial

• The model enforces multi-view geometric consistency by traversing semantic-visual-geometric cycles within autoregressive sequences.technical 80%
  Evidencepartial

  By traversing semantic-visual-geometric cycles (e.g., text to image to 3D to image) within autoregressive sequences, the model jointly enforces semantic alignment, appearance fidelity, and multi-view geometric consistency.

  Implicationpartial

  Described as a mechanism of the model's operation, though the specific improvement in consistency is implied rather than quantified here.

  Verificationpartial

  partial

• Experiments show that Omni123 significantly improves text-guided 3D generation and editing compared to existing methods.result 75%
  Evidencepartial

  Experiments show that Omni123 significantly improves text-guided 3D generation and editing

  Implicationpartial

  Claim of significant improvement is made, but the abstract does not provide specific metrics or comparison details.

  Verificationpartial

  partial

• The approach demonstrates a scalable path toward multimodal 3D world models.technical 70%
  Evidencepartial

  demonstrating a scalable path toward multimodal 3D world models.

  Implicationpartial

  Presented as a broader implication of the work, but is a forward-looking statement not directly proven by the results mentioned in the abstract.

  Verificationpartial

  partial

• Existing methods that rely on indirect 2D editing and 3D lifting pipelines sacrifice geometric consistency.limitation 85%
  Evidencepartial

  Existing methods often rely on indirect pipelines that edit in 2D and lift results into 3D via optimization, sacrificing geometric consistency.

  Implicationpartial

  Directly stated as a limitation of existing methods, forming the motivation for this work.

  Verificationpartial

  partial

• Extending native multimodal capability to 3D is challenging due to limited 3D data compared to abundant 2D imagery.technical 90%
  Evidencepartial

  extending such native capability to 3D remains challenging due to limited data. Compared to abundant 2D imagery, high-quality 3D assets are scarce, making 3D synthesis under-constrained.

  Implicationpartial

  Explicitly stated as the core problem motivating the research.

  Verificationpartial

  partial

Competitive landscape

Omni123 is a 3D-native foundation model that unifies text-to-2D and text-to-3D generation, leveraging 2D data as a prior to overcome 3D data scarcity.