HieraVid: Hierarchical Token Pruning for Fast Video Large Language Models

Dimensions overall score 7.0

• HieraVid achieves new state-of-the-art performance on video understanding benchmarks while retaining only 30% of tokensresult 95%
  Evidencepartial

  Remarkably, with only 30% of tokens retained, HieraVid achieves new state-of-the-art performance

  Implicationpartial

  Explicitly stated in abstract with specific numeric performance metrics

  Verificationpartial

  partial

• HieraVid maintains over 98% of the performance of LLaVA-Video-7Bresult 95%
  Evidencepartial

  while maintaining over 98% and 99% of the performance of LLaVA-Video-7B and LLaVA-OneVision-7B, respectively

  Implicationpartial

  Direct numeric claim stated in abstract with specific percentage

  Verificationpartial

  partial

• HieraVid maintains over 99% of the performance of LLaVA-OneVision-7Bresult 95%
  Evidencepartial

  while maintaining over 98% and 99% of the performance of LLaVA-Video-7B and LLaVA-OneVision-7B, respectively

  Implicationpartial

  Direct numeric claim stated in abstract with specific percentage

  Verificationpartial

  partial

• HieraVid uses a hierarchical pruning framework with three levels: segment-level, frame-level, and layer-levelmethod 90%
  Evidencepartial

  we decompose pruning into three levels: 1) segment-level, where video tokens are first temporally segmented and spatially merged; 2) frame-level, where similar frames within the same segment are jointly pruned to preserve diversity; 3) layer-level, redundancy gradually shrinks as LLM layer increases

  Implicationpartial

  Explicitly described in abstract with clear technical details

  Verificationpartial

  partial

• Existing video token pruning methods mainly prune at input level while neglecting inherent information structurelimitation 85%
  Evidencepartial

  Existing methods mainly prune video tokens at input level while neglecting the inherent information structure embedded in videos and large language models (LLMs)

  Implicationpartial

  Directly stated as motivation for the proposed method

  Verificationpartial

  partial

• HieraVid progressively and dynamically reduces visual redundancy in videosmethod 85%
  Evidencepartial

  we propose HieraVid, a hierarchical pruning framework that progressively and dynamically reduces visual redundancy

  Implicationpartial

  Directly stated as core functionality of the proposed method

  Verificationpartial

  partial

• The method is based on observations that videos possess segment-frame structure and LLMs propagate multi-modal information unidirectionallytechnical 80%
  Evidencepartial

  Based on two observations that videos possess the segment-frame structure and LLMs internally propagate multi-modal information unidirectionally

  Implicationpartial

  Directly stated as foundational observations for the method design

  Verificationpartial

  partial

• Extensive experiments were conducted on four widely used video understanding benchmarksresult 90%
  Evidencepartial

  We conduct extensive experiments on four widely used video understanding benchmarks to comprehensively evaluate the effectiveness of HieraVid

  Implicationpartial

  Explicitly stated in abstract with clear experimental scope

  Verificationpartial

  partial

Competitive landscape

HieraVid drastically reduces computational cost for Video LLMs by intelligently pruning video tokens hierarchically, achieving state-of-the-art performance with significantly fewer resources.