Optimizing Multi-Modal Models for Image-Based Shape Retrieval: The Role of Pre-Alignment and Hard Contrastive Learning

Dimensions overall score 8.0

• Pre-aligned image and shape encoders enable zero-shot image-based shape retrieval without requiring explicit view-based supervision.method 90%
  Evidencepartial

  we address IBSR through large-scale multi-modal pretraining and show that explicit view-based supervision is not required

  Implicationpartial

  Directly stated in abstract that explicit view-based supervision is not required and that pre-aligned encoders enable zero-shot retrieval

  Verificationpartial

  partial

• The proposed approach using pre-aligned encoders achieves state-of-the-art performance on shape retrieval metrics across multiple datasets.result 90%
  Evidencepartial

  Our evaluation demonstrates state-of-the-art performance, outperforming related methods on $Acc_{Top1}$ and $Acc_{Top10}$ for shape retrieval across multiple datasets

  Implicationpartial

  Explicitly stated in abstract with specific performance metrics mentioned

  Verificationpartial

  partial

• The OpenShape model combined with Point-BERT yields the best retrieval performance among evaluated pre-aligned encoders.result 85%
  Evidencepartial

  with best results observed for OpenShape combined with Point-BERT

  Implicationpartial

  Directly stated in abstract with specific model combination mentioned

  Verificationpartial

  partial

• Multi-modal hard contrastive loss (HCL) improves retrieval performance in standard instance retrieval tasks on shape-centric data.method 80%
  Evidencepartial

  training on our proposed multi-modal HCL yields dataset-dependent gains in standard instance retrieval tasks on shape-centric data

  Implicationpartial

  Strongly supported in abstract with mention of dataset-dependent gains

  Verificationpartial

  partial

• The proposed formulation enables cross-domain retrieval without retraining on the target database.technical 85%
  Evidencepartial

  This formulation allows skipping view synthesis and naturally enables zero-shot and cross-domain retrieval without retraining on the target database

  Implicationpartial

  Directly stated in abstract as a capability of the approach

  Verificationpartial

  partial

• The approach uses compact single-embedding shape descriptors for similarity search in retrieval.technical 90%
  Evidencepartial

  by embedding images and point clouds into a shared representation space and performing retrieval via similarity search over compact single-embedding shape descriptors

  Implicationpartial

  Explicitly described in abstract as part of the method

  Verificationpartial

  partial

• The method eliminates the need for view synthesis in image-based shape retrieval.technical 85%
  Evidencepartial

  This formulation allows skipping view synthesis

  Implicationpartial

  Directly stated as an advantage of the formulation in abstract

  Verificationpartial

  partial

Competitive landscape

Improve 3D model retrieval from images using pre-trained multi-modal encoders and hard contrastive learning, enabling zero-shot and cross-domain retrieval.