Evaluating Generative Models via One-Dimensional Code Distributions

Dimensions overall score 8.0

• Existing generative model evaluations using feature-distribution metrics like FID discard cues critical for perceptual quality.limitation 95%
  Evidencepartial

  Most evaluations of generative models rely on feature-distribution metrics such as FID, which operate on continuous recognition features that are explicitly trained to be invariant to appearance variations, and thus discard cues critical for perceptual quality.

  Implicationpartial

  The abstract explicitly states this as a motivation for the proposed method.

  Verificationpartial

  partial

• Modern 1D image tokenizers compactly encode both semantic and perceptual information in discrete visual tokens.technical 95%
  Evidencepartial

  where modern 1D image tokenizers compactly encode both semantic and perceptual information and quality manifests as predictable token statistics.

  Implicationpartial

  The abstract directly states this as a premise for their evaluation approach.

  Verificationpartial

  partial

• Codebook Histogram Distance (CHD) is a training-free distribution metric in token space.method 95%
  Evidencepartial

  We introduce Codebook Histogram Distance (CHD), a training-free distribution metric in token space

  Implicationpartial

  The abstract introduces CHD and explicitly states its characteristics.

  Verificationpartial

  partial

• Code Mixture Model Score (CMMS) is a no-reference quality metric learned from synthetic degradations of token sequences.method 95%
  Evidencepartial

  and Code Mixture Model Score (CMMS), a no-reference quality metric learned from synthetic degradations of token sequences.

  Implicationpartial

  The abstract introduces CMMS and explicitly states its characteristics.

  Verificationpartial

  partial

• VisForm is a benchmark comprising 210K images across 62 visual forms and 12 generative models.method 95%
  Evidencepartial

  we further propose VisForm, a benchmark of 210K images spanning 62 visual forms and 12 generative models with expert annotations.

  Implicationpartial

  The abstract provides specific details about the VisForm benchmark.

  Verificationpartial

  partial

• The proposed token-based metrics (CHD and CMMS) achieve state-of-the-art correlation with human judgments on AGIQA, HPDv2/3, and VisForm benchmarks.result 95%
  Evidencepartial

  Across AGIQA, HPDv2/3, and VisForm, our token-based metrics achieve state-of-the-art correlation with human judgments

  Implicationpartial

  The abstract directly states the performance of their metrics compared to human judgments across multiple benchmarks.

  Verificationpartial

  partial

• The authors will release all code and datasets to facilitate future research.method 95%
  Evidencepartial

  and we will release all code and datasets to facilitate future research.

  Implicationpartial

  The abstract explicitly states their intention to release resources.

  Verificationpartial

  partial

Competitive landscape

Evaluate generative models using discrete visual tokens for improved perceptual quality assessment, offering a training-free and no-reference approach with state-of-the-art correlation to human judgment.