MolmoPoint: Better Pointing for VLMs with Grounding Tokens

Dimensions overall score 7.0

• MolmoPoint sets a new state-of-the-art on image pointing with 70.7% accuracy on PointBench.result 95%
  Evidencepartial

  Using this method, we set a new state-of-the-art on image pointing (70.7% on PointBench)

  Implicationpartial

  Directly stated in the abstract with specific metric and benchmark name.

  Verificationpartial

  partial

• MolmoPoint-GUI-8B sets a new state-of-the-art among fully open models on GUI pointing with 61.1% accuracy on ScreenSpotPro.result 90%
  Evidencepartial

  set a new state-of-the-art among fully open models on GUI pointing (61.1% on ScreenSpotPro)

  Implicationpartial

  Directly stated in the abstract with specific model name, metric, and benchmark.

  Verificationpartial

  partial

• MolmoPoint improves video pointing with a 59.1% human preference win rate versus a text coordinate baseline.result 90%
  Evidencepartial

  improve video pointing (59.1% human preference win rate vs. a text coordinate baseline)

  Implicationpartial

  Directly stated in the abstract with specific comparative metric.

  Verificationpartial

  partial

• MolmoPoint's pointing mechanism directly selects visual tokens containing the target concept instead of generating coordinates, which is more intuitive and reduces token count.method 85%
  Evidencepartial

  Instead, we propose a more intuitive pointing mechanism that directly selects the visual tokens that contain the target concept.

  Implicationpartial

  Core method claim explicitly stated in the abstract and method description.

  Verificationpartial

  partial

• MolmoPoint uses a three-stage refinement process: a pointing token selects a coarse patch, a subpatch token refines within it, and a location token specifies the final point.technical 80%
  Evidencepartial

  To make this model more fine-grained, we follow these pointing tokens with an additional special token that selects a fine-grained subpatch within the initially selected region, and then a third token that specifies a location within that subpatch.

  Implicationpartial

  Method description clearly outlines the three-stage process in the abstract and Figure 1 caption.

  Verificationpartial

  partial

• The MolmoPoint-GUISyn dataset provides extremely dense annotations with 54 points per image on average, enabling efficient training via message trees.technical 85%
  Evidencepartial

  MolmoPoint-GUISyn also provides extremely dense annotations (54 points per image on average), making it very efficient to train on using message trees

  Implicationpartial

  Specific quantitative claim about dataset density with clear purpose stated.

  Verificationpartial

  partial

• MolmoPoint achieves higher sample efficiency compared to coordinate-based pointing methods.result 75%
  Evidencepartial

  We additionally show that our method achieves much higher sample efficiency

  Implicationpartial

  Claim is stated in the abstract but without specific quantitative comparison in provided excerpts.

  Verificationpartial

  partial

• Existing VLM pointing methods that generate coordinates require learning a complicated coordinate system and result in high token count.limitation 80%
  Evidencepartial

  VLMspointbygeneratingcoordinatesaspartoftheirtextoutput, whichrequireslearningacomplicated coordinate system and results in a high token count.

  Implicationpartial

  Problem statement clearly presented as motivation for the new method.

  Verificationpartial

  partial

Competitive landscape

A new pointing mechanism for VLMs that uses special tokens to directly select visual tokens, improving efficiency and state-of-the-art performance on image, GUI, and video pointing tasks.