MER-Bench: A Comprehensive Benchmark for Multimodal Meme Reappraisal

Dimensions overall score 8.0

• The paper introduces Meme Reappraisal, a novel multimodal generation task that transforms negatively framed memes into constructive ones while preserving their underlying scenario, entities, and structural layout.method 95%
  Evidencepartial

  we introduce Meme Reappraisal, a novel multimodal generation task that aims to transform negatively framed memes into constructive ones while preserving their underlying scenario, entities, and structural layout.

  Implicationpartial

  Directly and explicitly stated in the abstract as the core contribution of the paper.

  Verificationpartial

  partial

• The paper constructs MER-Bench, a benchmark of real-world memes with fine-grained multimodal annotations including source/target emotions, rewritten text, visual editing specs, and taxonomy labels.method 95%
  Evidencepartial

  To support this task, we construct MER-Bench, a benchmark of real-world memes with fine-grained multimodal annotations, including source and target emotions, positively rewritten meme text, visual editing specifications, and taxonomy labels covering visual type, sentiment polarity, and layout structure.

  Implicationpartial

  Directly and explicitly stated in the abstract as a key contribution.

  Verificationpartial

  partial

• The paper proposes a structured evaluation framework based on an MLLM-as-a-Judge paradigm, decomposing performance into modality-level generation quality, affect controllability, structural fidelity, and global affective alignment.method 90%
  Evidencepartial

  We further propose a structured evaluation framework based on a multimodal large language model (MLLM)-as-a-Judge paradigm, decomposing performance into modality-level generation quality, affect controllability, structural fidelity, and global affective alignment.

  Implicationpartial

  Directly stated in the abstract as a proposed method.

  Verificationpartial

  partial

• Extensive experiments across representative image-editing and multimodal-generation systems reveal substantial gaps in satisfying the constraints of structural preservation, semantic consistency, and affective transformation.result 90%
  Evidencepartial

  Extensive experiments across representative image-editing and multimodal-generation systems reveal substantial gaps in satisfying the constraints of structural preservation, semantic consistency, and affective transformation.

  Implicationpartial

  Directly stated in the abstract as a key result from experiments.

  Verificationpartial

  partial

• Meme Reappraisal requires emotion-controllable, structure-preserving multimodal transformation under multiple semantic and stylistic constraints, unlike prior works on meme understanding or generation.technical 85%
  Evidencepartial

  Unlike prior works on meme understanding or generation, Meme Reappraisal requires emotion-controllable, structure-preserving multimodal transformation under multiple semantic and stylistic constraints.

  Implicationpartial

  Directly stated in the abstract, distinguishing the task from prior work.

  Verificationpartial

  partial

• The proprietary dataset of annotated memes with fine-grained emotion, structure, and editing specs, plus domain-specific MLLM tuning, constitutes a moat for this research.technical 80%
  Evidencepartial

  Proprietary dataset of annotated memes with fine-grained emotion, structure, and editing specs, plus domain-specific MLLM tuning for meme reappraisal tasks.

  Implicationpartial

  Strongly implied in the analysis as the 'moat_source', directly linked to the constructed benchmark.

  Verificationpartial

  partial

• A risk of this approach is user backlash against automated 'positive' editing being perceived as censorship.limitation 85%
  Evidencepartial

  Specific risk 2: User backlash against automated 'positive' editing seen as censorship

  Implicationpartial

  Explicitly listed as a specific risk in the analysis section.

  Verificationpartial

  partial

• There is a risk of bias in emotion detection across cultures or contexts, which is a caveat of the approach.limitation 85%
  Evidencepartial

  Risk of bias in emotion detection across cultures or contexts

  Implicationpartial

  Explicitly listed as a caveat in the analysis section.

  Verificationpartial

  partial

Competitive landscape

MER-Bench enables the transformation of negative memes into constructive ones through emotion-controllable multimodal generation.