DisaBench: A Participatory Evaluation Framework for Disability Harms in Language Models

Dimensions overall score 7.0

• A participatory framework and dataset for evaluating disability-related harms in language models, designed for integration into existing safety pipelines. We introduce DisaBench: a taxonomy of twelve disability harm categories co-created with people with disabilities and red teaming experts, a taxonomy-driven evaluation methodology that pairs benign and adversarial prompts across seven life domains, and a dataset of 175 prompts with human-annotated labels on 525 prompt-response pairs.problem 35%
  Evidencepartial

  A participatory framework and dataset for evaluating disability-related harms in language models, designed for integration into existing safety pipelines. We introduce DisaBench: a taxonomy of twelve disability harm categories co-created with people with disabilities and red teaming experts, a taxonomy-driven evaluation methodology that pairs benign and adversarial prompts across seven life domains, and a dataset of 175 prompts with human-annotated labels on 525 prompt-response pairs.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• General-purpose safety benchmarks for large language models do not adequately evaluate disability-related harms. We introduce DisaBench: a taxonomy of twelve disability harm categories co-created with people with disabilities and red teaming experts, a taxonomy-driven evaluation methodology that pairs benign and adversarial prompts across seven life domains, and a dataset of 175 prompts with human-annotated labels on 525 prompt-response pairs.method 35%
  Evidencepartial

  General-purpose safety benchmarks for large language models do not adequately evaluate disability-related harms. We introduce DisaBench: a taxonomy of twelve disability harm categories co-created with people with disabilities and red teaming experts, a taxonomy-driven evaluation methodology that pairs benign and adversarial prompts across seven life domains, and a dataset of 175 prompts with human-annotated labels on 525 prompt-response pairs.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• ScienceToStartup currently rates this 7.0/10 on the public viability pass. We will release the dataset, taxonomy, and methodology via Hugging Face and an open-source red teaming framework for direct integration into existing safety pipelines with no additional infrastructure. A public repository is linked, so build verification can inspect implementation evidence instead of treating the paper as PDF-only.result 35%
  Evidencepartial

  ScienceToStartup currently rates this 7.0/10 on the public viability pass. We will release the dataset, taxonomy, and methodology via Hugging Face and an open-source red teaming framework for direct integration into existing safety pipelines with no additional infrastructure. A public repository is linked, so build verification can inspect implementation evidence instead of treating the paper as PDF-only.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• LLM Safety & Evaluation moved forward this cycle; last verified May 2026. Public score 7.0/10. Implementation evidence is present through a linked repository.why now 35%
  Evidencepartial

  LLM Safety & Evaluation moved forward this cycle; last verified May 2026. Public score 7.0/10. Implementation evidence is present through a linked repository.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

Competitive landscape

A participatory framework and dataset for evaluating disability-related harms in language models, designed for integration into existing safety pipelines.