ClimateCheck 2026: Scientific Fact-Checking and Disinformation Narrative Classification of Climate-related Claims

Dimensions overall score 4.0

• ClimateCheck 2026 expanded its training data by approximately three times compared to the previous iteration.method 95%
  Evidencepartial

  we address these challenges and substantially expand the scope of our dataset. We release approx. three times more training data

  Implicationpartial

  The abstract explicitly states the expansion of training data.

  Verificationpartial

  partial

• ClimateCheck 2026 introduced a new task for classifying disinformation narratives.method 95%
  Evidencepartial

  and introduce a new task,Disinformation Narrative Classification, which moves beyond claim-level verif

  Implicationpartial

  The abstract clearly states the introduction of a new task.

  Verificationpartial

  partial

• The inter-annotator agreement (IAA) for the training split of ClimateCheck 2026, including extensions, was Cohen’s κ = 0.73.method 95%
  Evidencepartial

  The overall inter-annotator agreement (IAA) of the training split including the extension is Cohen’sκ = 0.73, indicating substan-tial agreement

  Implicationpartial

  This is a specific metric reported directly in the text.

  Verificationpartial

  partial

• The best checkpoint for a specific model achieved a macro-F1 of 0.886 and an accuracy of 0.921 on the combined validation set.result 95%
  Evidencepartial

  The selected checkpoint at step 26,000 (epoch 1.7) achieves a macro-F1 of 0.886 and an accuracy of 0.921 on the combined validation set

  Implicationpartial

  Specific performance metrics are provided for a selected checkpoint.

  Verificationpartial

  partial

• The best performing system among the 2026 submissions for Task 1.2 achieved a weighted F1 score of 0.670.result 90%
  Evidencepartial

  DFKI-IML (Liang et al., 2026) 0.193 0.352 0.401 0.377 0.670 0.641 0.620 0.972

  Implicationpartial

  This is a specific result reported in Table 2, with the best result bolded.

  Verificationpartial

  partial

• The competition attracted 20 registered participants and 8 leaderboard submissions.market 95%
  Evidencepartial

  the competition attracted 20 registered participants and 8 leaderboard submissions

  Implicationpartial

  This information is directly stated in the abstract.

  Verificationpartial

  partial

• Systems participating in ClimateCheck 2026 combined dense retrieval pipelines, cross-encoder ensembles, and large language models with structured hierarchical reasoning.technical 95%
  Evidencepartial

  with systems combining dense retrieval pipelines, cross-encoder ensembles, and large language models with structured hierarchical reasoning.

  Implicationpartial

  The abstract describes the common system architectures used by participants.

  Verificationpartial

  partial

• Conventional metrics for ranking systems may exhibit systematic biases when assessing retrieval quality under incomplete annotations.limitation 90%
  Evidencepartial

  we adapt an automated framework to assess retrieval quality under incomplete annotations, exposing systematic biases in how conventional metrics rank systems.

  Implicationpartial

  The abstract highlights a limitation and a finding related to evaluation metrics.

  Verificationpartial

  partial

Competitive landscape

A shared task competition focused on developing AI systems for scientific fact-checking and disinformation classification of climate-related claims.