Finding Distributed Object-Centric Properties in Self-Supervised Transformers

Dimensions overall score 7.0

GitHub Code Pulse

No public code linked for this paper yet.

• Object-centric properties are encoded in the similarity maps derived from all three attention components (query, key, and value).result 95%
• Object-centric information is distributed across the network, not just confined to the final layer.result 95%
• Object-DINO is a training-free method that extracts distributed object-centric information by clustering attention heads.method 95%
• Applying Object-DINO to DINO-V3 improves CorLoc by +4.8, +5.7, and +3.6 points on VOC 2007, VOC 2012, and COCO 20k, respectively.result 95%
• Applying Object-DINO to DINO-V2 improves CorLoc by +9.5, +12.4, and +7.8 points on VOC 2007, VOC 2012, and COCO 20k, respectively.result 95%
• Object-DINO provides explicit visual grounding to reduce object hallucination in Multimodal Large Language Models (MLLMs).result 95%
• The proposed decoding strategy using Object-DINO achieves the highest Precision and F1-score on the POPE benchmark across LLaVA-1.5, Instruct-BLIP, and Qwen-VL.result 95%
• The [CLS] token, trained on an image-level objective, summarizes the entire image, leading to spurious activations and poor object localization.limitation 90%
• Object-centric properties are encoded in similarity maps derived from query, key, and value components of patch-level attention.result 95%
• Object-centric information is distributed across all layers of the network, not just the final layer.result 95%
• Object-DINO is a training-free method that extracts distributed object-centric information by clustering attention heads.method 95%
• Object-DINO improves unsupervised object discovery performance, achieving up to +12.4 CorLoc gains on DINO-V2.result 95%

Competitive landscape

Competitor map is still being generated for this paper. Enable generation or check back soon.

Startup potential card