VistaGEN: Consistent Driving Video Generation with Fine-Grained Control Using Multiview Visual-Language Reasoning

Dimensions overall score 4.0

• VistaGEN enables fine-grained object-level controllability for diverse driving videos while preserving spatiotemporal consistency in long video generation.method 90%
  Evidencepartial

  In this paper, we present a new driving video generation technique, called VistaGEN, which enables fine-grained control of specific entities, including 3D objects, images, and text descriptions, while maintaining spatiotemporal consistency in long video sequences.

  Implicationpartial

  Explicitly stated as the core contribution in the abstract and title.

  Verificationpartial

  partial

• Incorporating a local object prompt (c_local) significantly improves semantic alignment (text and image) in multiview video generation compared to using only a global prompt.result 95%
  Evidencepartial

  While geometric accuracy (mAP) remains high in both settings due to box constraints, semantic alignment improves significantly with c_local.

  Implicationpartial

  Directly supported by quantitative results in Table 3, showing a large increase in alignment scores.

  Verificationpartial

  partial

• The proposed generation-evaluation-regeneration closed-loop mechanism, using a Multiview Vision-Language Model (MV-VLM) evaluator, enables preservation of content consistency in long-range video sequences.method 90%
  Evidencepartial

  This results in a novel generation-evaluation-regeneration closed-loop mechanism, enabling the preservation of the content consistency during the long-range video sequences.

  Implicationpartial

  Explicitly stated as a key innovation and the mechanism is described in detail.

  Verificationpartial

  partial

• VistaGEN's hierarchical condition injection mechanism decomposes control into macro-level global scene control and micro-level fine-grained object control.technical 85%
  Evidencepartial

  We decompose the control conditions C into Macro-level Global Scene Control c_global and Micro-level Fine-grained Object Control c_local.

  Implicationpartial

  Directly described in the method section (Section 3).

  Verificationpartial

  partial

• The object-level refinement module uses explicit 3D geometric cues to improve object-level spatiotemporal coherence within the closed-loop generation.method 85%
  Evidencepartial

  Besides, we also build up an object-level refinement module, which uses explicit 3D geometric cues to improve the object-level spatio-temporal coherence within the closed-up loop generation.

  Implicationpartial

  Directly stated as a component of the proposed system.

  Verificationpartial

  partial

• Previous driving video generation approaches highly rely on structure prompts and lack effective fine-grained controllability for object-level manipulation.limitation 80%
  Evidencepartial

  However, most of the previous driving video generation approaches highly rely on structure prompts (such as BEV, 3D boxes, HDMaps, and optical flow), without an effective ability for fine-grained controllability of object-level manipulation.

  Implicationpartial

  Directly stated as a limitation of prior work in the analysis.

  Verificationpartial

  partial

• The MV-VLM evaluator is instantiated using a 'Dual-Stream Perception, Unified Reasoning' paradigm built upon the Qwen-V3 architecture.technical 90%
  Evidencepartial

  We instantiate the intelligent evaluator E using a 'Dual-Stream Perception, Unified Reasoning' paradigm built upon the Qwen-V3 [68] architecture.

  Implicationpartial

  Specific technical detail directly provided in the method description.

  Verificationpartial

  partial

• Extensive evaluation shows VistaGEN achieves diverse driving video generation with fine-grained controllability, especially for long-tail objects, and better spatiotemporal consistency than previous approaches.result 85%
  Evidencepartial

  Extensive evaluation shows that our VistaGEN achieves diverse driving video generation results with fine-grained controllability, especially for long-tail objects, and much better spatiotemporal consistency than previous approaches.

  Implicationpartial

  Claim is made in the abstract, though specific comparative results are not quoted in the provided excerpts.

  Verificationpartial

  partial

Competitive landscape

A novel technique for generating driving videos with fine-grained object-level control and improved spatiotemporal consistency.