Rethinking UMM Visual Generation: Masked Modeling for Efficient Image-Only Pre-training

Dimensions overall score 9.0

• The proposed Image-Only Training for UMMs (IOMM) framework improves training efficiency by using abundant unlabeled image-only data for pre-training, removing dependency on paired data for this phase.method 95%
• IOMM-B (3.6B) model achieves state-of-the-art performance with 0.89 on GenEval and 0.55 on WISE, surpassing BAGEL-7B (0.82 & 0.55) and BLIP3-o-4B (0.84 & 0.50).result 95%
• IOMM-B (3.6B) model was trained from scratch using only approximately 1050 H800 GPU hours, with 1000 hours dedicated to the efficient image-only pre-training stage.technical 95%
• The second stage fine-tunes the model using a mixture of unlabeled images and a small curated set of text-image pairs, leading to improved instruction alignment and generative quality.method 90%
• Unified Multimodal Models (UMMs) are constrained by inefficient pre-training paradigms and scarce, high-quality text-image paired data for their visual generation components.limitation 90%
• The IOMM framework addresses bottlenecks in UMM visual generation pre-training by using a data-efficient two-stage training approach.technical 90%
• Extensive experiments show that IOMM not only improves training efficiency but also achieves state-of-the-art (SOTA) performance.result 90%
• Image-Only Training for UMMs (IOMM) is a data-efficient two-stage training framework for Unified Multimodal Models (UMMs) visual generation components.method 70%
• The first stage of IOMM pre-trains the visual generative component exclusively using abundant unlabeled image-only data.method 70%
• IOMM removes the dependency on paired text-image data for the costly pre-training phase of UMM visual generation.method 70%
• The second stage of IOMM fine-tunes the model using a mixture of unlabeled images and a small curated set of text-image pairs.method 70%
• The IOMM-B (3.6B) model achieved a GenEval score of 0.89.result 70%

Competitive landscape

Competitor map is still being generated for this paper. Enable generation or check back soon.

Startup potential card