Local-Global Prompt Learning via Sparse Optimal Transport

Dimensions overall score 8.0

GitHub Code Pulse

No public code linked for this paper yet.

• SOT-GLP achieves 85.1% average accuracy on the standard 11-dataset benchmark with 16-shot ViT-B/16, outperforming prior prompt-learning methods.result 95%
  Evidencepartial

  On the standard 11-dataset benchmark with 16-shot ViT-B/16, SOT-GLP achieves 85.1% average accuracy, outperforming prior prompt-learning methods.

  Implicationpartial

  Directly stated in abstract with specific numeric results

  Verificationpartial

  partial

• SOT-GLP yields state-of-the-art OOD detection performance with 94.2% AUC, surpassing fully adapted models.result 95%
  Evidencepartial

  yielding state-of-the-art OOD detection performance (94.2% AUC) that surpasses fully adapted models.

  Implicationpartial

  Directly stated in abstract with specific numeric results

  Verificationpartial

  partial

• SOT-GLP introduces a shared sparse patch support and balanced optimal transport allocation to partition salient visual regions among class-specific local prompts.method 90%
  Evidencepartial

  We propose SOT-GLP, which introduces a shared sparse patch support and balanced optimal transport allocation to explicitly partition salient visual regions among class-specific local prompts while preserving global alignment.

  Implicationpartial

  Directly stated in abstract as core methodological contribution

  Verificationpartial

  partial

• The method learns both shared global prompts and class-specific local prompts, with the global branch maintaining standard image-text matching for robust category-level alignment.method 90%
  Evidencepartial

  Our method learns shared global prompts and class-specific local prompts. The global branch maintains standard image-text matching for robust category-level alignment.

  Implicationpartial

  Directly stated in abstract describing the method's architecture

  Verificationpartial

  partial

• The local branch constructs a class-conditioned sparse patch set using V-V attention and aligns it to multiple class-specific prompts via balanced entropic optimal transport.technical 90%
  Evidencepartial

  The local branch constructs a class-conditioned sparse patch set using V-V attention and aligns it to multiple class-specific prompts via balanced entropic optimal transport, yielding a soft partition of patches that prevents prompt overlap and collapse.

  Implicationpartial

  Directly stated in abstract describing technical implementation

  Verificationpartial

  partial

• There exists a distinct accuracy-robustness trade-off in prompt learning where learnable projections optimize in-distribution fit but alter the foundational feature space.technical 85%
  Evidencepartial

  We identify a distinct accuracy-robustness trade-off in prompt learning: while learnable projections optimize in-distribution fit, they alter the foundational feature space.

  Implicationpartial

  Directly stated in abstract as an identified phenomenon

  Verificationpartial

  partial

• A projection-free local alignment preserves the native geometry of the CLIP manifold, contributing to superior OOD detection performance.technical 85%
  Evidencepartial

  We demonstrate that a projection-free local alignment preserves the native geometry of the CLIP manifold, yielding state-of-the-art OOD detection performance

  Implicationpartial

  Directly stated in abstract as a technical advantage

  Verificationpartial

  partial

• Recent approaches that incorporate local image-text alignment often select local regions independently for each prompt, leading to redundant local feature usage and prompt overlap.limitation 80%
  Evidencepartial

  Recent works extend this paradigm by incorporating local image-text alignment to capture fine-grained visual cues, yet these approaches often select local regions independently for each prompt, leading to redundant local feature usage and prompt overlap.

  Implicationpartial

  Directly stated in abstract as limitation of prior work

  Verificationpartial

  partial

Startup potential card