Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning

Dimensions overall score 8.0

GitHub Code Pulse

No public code linked for this paper yet.

• Cosmos Policy adapts a pretrained video model (Cosmos-Predict2) into a robot policy through a single stage of post-training without architectural modifications.method 70%
• Cosmos Policy directly generates robot actions encoded as latent frames within the video model's latent diffusion process.technical 70%
• Cosmos Policy generates future state images and values (expected cumulative rewards) encoded as latent frames, enabling test-time planning of action trajectories.technical 70%
• Cosmos Policy achieves state-of-the-art performance on the LIBERO simulation benchmark with a 98.5% average success rate.result 70%
• Cosmos Policy achieves state-of-the-art performance on the RoboCasa simulation benchmark with a 67.1% average success rate.result 70%
• Cosmos Policy achieves the highest average score in challenging real-world bimanual manipulation tasks, outperforming strong diffusion policies, video model-based policies, and vision-language-action models.result 70%
• Reliance on specific pretrained models might limit Cosmos Policy's adaptability to drastically different environments.limitation 70%
• Market opportunity exists in industrial robotics for enhanced automation capabilities and predictive control.market 70%

Competitive landscape

Competitor map is still being generated for this paper. Enable generation or check back soon.

Startup potential card