AMIGO: Agentic Multi-Image Grounding Oracle Benchmark

Dimensions overall score 7.0

GitHub Code Pulse

No public code linked for this paper yet.

• AMIGO is a long-horizon benchmark for hidden-target identification over galleries of visually similar images.method 95%
• AMIGO evaluates agentic VLMs on their ability to actively acquire missing information through constrained questioning, unlike benchmarks that assess solving presented problems.method 85%
• The AMIGO protocol uses a constrained action space where the model asks attribute-focused Yes/No/Unsure questions and is penalized for invalid actions with Skip.method 90%
• AMIGO supports controlled oracle imperfections to probe model robustness and verification behavior under inconsistent feedback.method 90%
• AMIGO is instantiated with a 'Guess My Preferred Dress' task.method 95%
• AMIGO evaluates interaction quality via metrics like evidence verification, efficiency, protocol compliance, and noise tolerance, not just final success.method 85%
• AMIGO differs from related interactive clarification benchmarks by focusing on a closed set of visually similar images and evaluating the interaction policy via trajectory-level signals.technical 80%
• Most current evaluations for agentic vision-language models still focus on single-image, single-turn correctness, which is a limitation AMIGO addresses.limitation 90%

Competitive landscape

Competitor map is still being generated for this paper. Enable generation or check back soon.

Startup potential card