Frequency-Modulated Visual Restoration for Matryoshka Large Multimodal Models

Dimensions overall score 8.0

GitHub Code Pulse

No public code linked for this paper yet.

• FMVR reduces the FLOPs of LLaVA-1.5-7B by 89% while maintaining almost 100% of the original accuracy.result 90%
• FMVR disentangles visual representation into low- and high-frequency components using AvgPool and MaxPool operations.method 85%
• The high-frequency component from AvgPool acts as a saliency filter to enhance salient visual semantics.technical 80%
• The low-frequency component from MaxPool acts as an anti-saliency filter to strengthen weak visual semantics.technical 80%
• FMVR enables preservation of visual semantics dominated by few visual tokens and restoration of diluted visual semantics.method 75%
• FMVR is a plug-and-play and extremely simple strategy for boosting LMM reasoning under visual token reduction.method 85%
• Previous methods for reducing visual tokens inevitably result in loss of visual semantic information.limitation 70%
• FMVR enables elastic adjustment of visual token count during inference while maintaining comparable performance.technical 75%

Competitive landscape

Competitor map is still being generated for this paper. Enable generation or check back soon.

Startup potential card