CDH-Bench: A Commonsense-Driven Hallucination Benchmark for Evaluating Visual Fidelity in Vision-Language Models

Dimensions overall score 7.0

• Current vision-language model benchmarks are poorly suited to detect commonsense-driven hallucination because they use commonsense-consistent imagery where visual evidence and priors typically agree.limitation 90%
  Evidencepartial

  Most benchmarks use commonsense-consistent imagery, so visual evidence and commonsense priors typically agree.

  Implicationpartial

  Directly stated in the abstract and analysis as the motivation for creating CDH-Bench.

  Verificationpartial

  partial

• CDH-Bench is a benchmark designed to create explicit visual evidence-commonsense conflicts across three dimensions: counting anomalies, relational anomalies, and attribute anomalies.method 95%
  Evidencepartial

  To evaluate it, we introduce CDH-Bench, a benchmark designed to create explicit visual evidence–commonsense conflicts. CDH-Bench covers three dimensions: counting anomalies, relational anomalies, and attribute anomalies.

  Implicationpartial

  Explicitly stated in the abstract and title as the core contribution of the paper.

  Verificationpartial

  partial

• Even strong vision-language models remain vulnerable to prior-driven normalization under visual evidence-commonsense conflict.result 85%
  Evidencepartial

  Results show that even strong models remain vulnerable to prior-driven normalization under visual evidence–commonsense conflict.

  Implicationpartial

  Directly stated in the abstract as a key finding from evaluating frontier models.

  Verificationpartial

  partial

• The Commonsense Collapse Rate (CCR) metric provides a sharper diagnostic signal than accuracy alone by quantifying how often models revert to commonsense alternatives when they fail.method 80%
  Evidencepartial

  This distinction, quantified through CCR, provides a sharper diagnostic signal than accuracy alone, where direct answer competition makes the interpretation most transparent.

  Implicationpartial

  Strongly supported by the analysis describing CCR's purpose and advantage over standard accuracy.

  Verificationpartial

  partial

• CDH-Bench contains 600 images organized as 300 counterfactual-commonsense image pairs, yielding 1,200 evaluated instances across different conditions and formats.Source proofmethod 90%
  Evidenceverified

  We construct 600 images, organized as 300 counterfactual and CS images... yielding 300×2×2 = 1,200 evaluated instances in total.

  Implicationpartial

  Specific numeric details are provided in the analysis section.

  Verificationverified

  verified

• Counterfactual Accuracy (CF-Acc) is the primary measure of visual fidelity under visual evidence-commonsense conflict.method 85%
  Evidencepartial

  CF-Acc is the accuracy on counterfactual (CF) images, and is our primary measure of visual fidelity under conflict.

  Implicationpartial

  Explicitly stated in the metrics description section.

  Verificationpartial

  partial

• Commonsense-driven hallucination matters most in domains like medical imaging, quality inspection, scientific discovery, and forensics where anomalies are critical.market 80%
  Evidencepartial

  CDH matters most where anomalies matter: medical imaging, quality inspection, scientific discovery, and forensics.

  Implicationpartial

  Directly stated in the analysis with specific domain examples.

  Verificationpartial

  partial

• The Relative Prior Dependency (RPD) metric measures how much capability is lost when visual evidence conflicts with commonsense, normalized by baseline performance.method 75%
  Evidencepartial

  RPD also answers Q2, but in relative terms: of what the model can do when visual evidence and commonsense agree, how much is lost when they conflict?

  Implicationpartial

  Supported by the description of RPD's purpose and calculation method.

  Verificationpartial

  partial

Competitive landscape

A new benchmark to diagnose and mitigate commonsense-driven hallucinations in vision-language models, improving their visual fidelity.