ARXIV:2605.30561 · 3D VISION WITH VLMS · SUBMITTED 01 JUN · 20:30 UTC · FRESHNESS STALE

VerifiedSource: PDF linkedVerifiedPaperPack: citation fields availablePartialProof: unverified proof status

VLM3: Vision Language Models Are Native 3D Learners

Zhipeng Cai · Zhuang Liu · Yunyang Xiong · Zechun Liu · Vikas Chandra · Yangyang Shi · arXiv

This paper proposes a simple yet effective method to enable standard Vision Language Models to master diverse 3D tasks, advancing accuracy and enabling new applications.

Ship in 2-4 weeks›Score4.0Evidence unverified

Opportunity summary

Pain This paper proposes a simple yet effective method to enable standard Vision Language Models to master diverse 3D tasks, advancing accuracy and enabling new applications.

Evidence 0 refs | 4 sources | 67% coverage

Blocker Evidence unverified

Open Build Read PDF Signal Canvas Track

PROBLEM

This paper proposes a simple yet effective method to enable standard Vision Language Models to master diverse 3D tasks, advancing accuracy and enabling new applications. They have shown promising performance in semantic understanding.

METHOD

Vision Language Models (VLMs) enable a unified model to solve various vision tasks through prompting. They have shown promising performance in semantic understanding.

Full abstract

Vision Language Models (VLMs) enable a unified model to solve various vision tasks through prompting. They have shown promising performance in semantic understanding. However, 3D understanding still largely relies on expert vision models with complex task-specific designs. The key argument this work wants to make is that VLMs are native 3D learners. Our in-depth large scale study shows that 1) focal length unification, 2) text-based pixel reference and 3) data mixture and scaling, are all you need for effective 3D learning. Model architecture changes, large models, heavy data augmentations, and complex losses including the regression formulation, many of which form the foundation of expert vision models, are actually not necessary conditions. As a result, we propose VLM3, a scalable method with the simplest design that enables standard VLMs to master diverse 3D tasks. VLM3 not only advances the VLM depth estimation accuracy by a large margin (0.84 -> 0.9), but also enables diverse 3D tasks such as pixel correspondence, camera pose estimation and object-level 3D understanding, matching expert vision model accuracy while maintaining standard architectures and text-based training. We believe VLM3 opens up a new paradigm for simple and scalable 3D learning.

RESULT

ScienceToStartup currently rates this 4.0/10 on the public viability pass. Vision Language Models (VLMs) enable a unified model to solve various vision tasks through prompting. A public repository is linked, so build verification can…

WHY NOW

3D Vision with VLMs moved forward this cycle; last verified June 2026. Public score 4.0/10. Implementation evidence is present through a linked repository.

Continue into Read for claims, analysis, references, and neighboring papers.

Opportunity summary

Score4.0

PainThis paper proposes a simple yet effective method to enable standard Vision Language Models to master diverse 3D tasks, advancing accuracy and enabling new applications.

Evidence0 refs | 4 sources | 67% coverage

Blockerno shell-level blocker reported

Analysis summary

This paper proposes a simple yet effective method to enable standard Vision Language Models to master diverse 3D tasks, advancing accuracy and enabling new applications.

VerifiedSource: PDF linkedVerifiedPaperPack: citation fields availablePartialProof: unverified proof status

Competitive landscape

This paper proposes a simple yet effective method to enable standard Vision Language Models to master diverse 3D tasks, advancing accuracy and enabling new applications.

Segment

3D Vision with VLMs

Adoption evidence

Public code linked for build inspection

Commercial read

4.0/10 public viability

Direct

not classified

Adjacent

not classified

Substitute

not classified

Unknown

not classified

{ "contract_version": "paper-r2", "paper_id": "58a75532-9b49-4554-b213-81b13dea1e83", "arxiv_id": "2605.30561", "canonical_route": "/paper/vlm3-vision-language-models-are-native-3d-learners", "active_tab": "synced from current hash by the drawer client", "selected_artifact": "vlm3-vision-language-models-are-native-3d-learners", "endpoints": { "paper_pack": "/api/v1/paper/vlm3-vision-language-models-are-native-3d-learners/paper-pack", "build_passport": "/api/v1/paper/vlm3-vision-language-models-are-native-3d-learners/build-passport", "mcp_resource": "sciencetostartup://surfaces/paper-workspace" } }

{ "surface": "paper", "mode": "paper", "query": "VLM3: Vision Language Models Are Native 3D Learners", "normalized_query": "2605.30561", "route": "/paper/vlm3-vision-language-models-are-native-3d-learners", "paper_ref": "vlm3-vision-language-models-are-native-3d-learners", "topic_slug": null, "benchmark_ref": null, "dataset_ref": null }

{ "@context": "https://schema.org", "@graph": [ { "@type": "WebPage", "@id": "https://sciencetostartup.com/paper/vlm3-vision-language-models-are-native-3d-learners#webpage", "url": "https://sciencetostartup.com/paper/vlm3-vision-language-models-are-native-3d-learners", "name": "VLM3: Vision Language Models Are Native 3D Learners", "description": "This paper proposes a simple yet effective method to enable standard Vision Language Models to master diverse 3D tasks, advancing accuracy and enabling new applications.", "isPartOf": { "@id": "https://sciencetostartup.com/#website" } }, { "@type": "ScholarlyArticle", "@id": "https://sciencetostartup.com/paper/vlm3-vision-language-models-are-native-3d-learners#scholarlyArticle", "headline": "VLM3: Vision Language Models Are Native 3D Learners", "description": "This paper proposes a simple yet effective method to enable standard Vision Language Models to master diverse 3D tasks, advancing accuracy and enabling new applications.", "url": "https://sciencetostartup.com/paper/vlm3-vision-language-models-are-native-3d-learners", "sameAs": "https://arxiv.org/abs/2605.30561", "identifier": { "@type": "PropertyValue", "propertyID": "arXiv", "value": "2605.30561" }, "isAccessibleForFree": true, "isPartOf": { "@id": "https://sciencetostartup.com/#website" }, "datePublished": "2026-05-28T20:48:55.000Z", "author": [ { "@type": "Person", "name": "Zhipeng Cai" }, { "@type": "Person", "name": "Zhuang Liu" }, { "@type": "Person", "name": "Yunyang Xiong" }, { "@type": "Person", "name": "Zechun Liu" }, { "@type": "Person", "name": "Vikas Chandra" }, { "@type": "Person", "name": "Yangyang Shi" } ], "codeRepository": "https://github.com/facebookresearch/VLM3", "additionalProperty": [ { "@type": "PropertyValue", "propertyID": "viabilityScore", "value": 4 }, { "@type": "PropertyValue", "propertyID": "researchDomain", "value": "3D Vision with VLMs" }, { "@type": "PropertyValue", "propertyID": "commercialReadiness", "value": "code, repo url" } ] }, { "@type": "SoftwareSourceCode", "@id": "https://sciencetostartup.com/paper/vlm3-vision-language-models-are-native-3d-learners#software", "name": "VLM3: Vision Language Models Are Native 3D Learners - Source Code", "description": "This paper proposes a simple yet effective method to enable standard Vision Language Models to master diverse 3D tasks, advancing accuracy and enabling new applications.", "codeRepository": "https://github.com/facebookresearch/VLM3", "url": "https://github.com/facebookresearch/VLM3" }, { "@type": "BreadcrumbList", "itemListElement": [ { "@type": "ListItem", "position": 1, "name": "Home", "item": "https://sciencetostartup.com" }, { "@type": "ListItem", "position": 2, "name": "3D Vision with VLMs", "item": "https://sciencetostartup.com/topics" }, { "@type": "ListItem", "position": 3, "name": "VLM3: Vision Language Models Are Native 3D Learners", "item": "https://sciencetostartup.com/paper/vlm3-vision-language-models-are-native-3d-learners" } ] } ] }

Competitive landscape

This paper proposes a simple yet effective method to enable standard Vision Language Models to master diverse 3D tasks, advancing accuracy and enabling new applications.

Segment

3D Vision with VLMs

Adoption evidence

Public code linked for build inspection

Commercial read

4.0/10 public viability

Direct

not classified

Adjacent

not classified

Substitute

not classified

Unknown

not classified

VLM3: Vision Language Models Are Native 3D Learners

VLM3: Vision Language Models Are Native 3D Learners

Claim map

Constellation map

Competitive landscape

Buzz

PDF

REFERENCES

Related Papers

Subscribe to the weekly brief

Build artifacts

Brief

Experiment plan

Validation checklist

Scientific founder

Translational engineer

Domain operator

GTM lead

Regulatory/clinical advisor

Timeline

Claim map

Constellation map

Competitive landscape

Buzz

PDF

REFERENCES

Related Papers

Subscribe to the weekly brief

Build artifacts

Brief

Experiment plan

Validation checklist

Scientific founder

Translational engineer

Domain operator

GTM lead

Regulatory/clinical advisor

Timeline