AstroConcepts: A Large-Scale Multi-Label Classification Corpus for Astrophysics

Dimensions overall score 4.0

• AstroConcepts is a corpus of 21,702 English astrophysics abstracts labeled with 2,367 concepts from the Unified Astronomy Thesaurus.method 95%
  Evidencepartial

  We introduce AstroConcepts, a corpus of English abstracts from 21,702 published astrophysics papers, labeled with 2,367 concepts from the Unified Astronomy Thesaurus.

  Implicationpartial

  Directly stated in the abstract with specific numbers

  Verificationpartial

  partial

• The AstroConcepts corpus exhibits severe label imbalance with 76% of concepts having fewer than 50 training examples.result 95%
  Evidencepartial

  The corpus exhibits severe label imbalance, with 76% of concepts having fewer than 50 training examples.

  Implicationpartial

  Directly stated in the abstract with specific percentage

  Verificationpartial

  partial

• Vocabulary-constrained LLMs achieve competitive performance relative to domain-adapted models in astrophysics classification.result 90%
  Evidencepartial

  First, vocabulary-constrained LLMs achieve competitive performance relative to domain-adapted models in astrophysics classification, suggesting a potential for parameter-efficient approaches.

  Implicationpartial

  Directly stated as a key finding in the abstract

  Verificationpartial

  partial

• Domain adaptation yields relatively larger improvements for rare, specialized terminology compared to common concepts.result 85%
  Evidencepartial

  Second, domain adaptation yields relatively larger improvements for rare, specialized terminology, although absolute performance remains limited across all methods.

  Implicationpartial

  Directly stated in the abstract but qualified with 'relatively larger improvements'

  Verificationpartial

  partial

• Absolute performance remains limited across all methods for rare, specialized terminology in astrophysics classification.limitation 85%
  Evidencepartial

  although absolute performance remains limited across all methods.

  Implicationpartial

  Directly stated in the abstract as a limitation of current methods

  Verificationpartial

  partial

• Frequency-stratified evaluation reveals performance patterns that are hidden by aggregate scores in scientific multi-label classification.method 90%
  Evidencepartial

  Third, we propose frequency-stratified evaluation to reveal performance patterns that are hidden by aggregate scores, thereby making robustness assessment central to scientific multi-label evaluation.

  Implicationpartial

  Directly stated as a proposed method and key finding in the abstract

  Verificationpartial

  partial

• Existing scientific corpora lack comprehensive controlled vocabularies and focus on broad categories rather than systematic study of extreme imbalance.limitation 80%
  Evidencepartial

  Existing scientific corpora lack comprehensive controlled vocabularies, focusing instead on broad categories and limiting systematic study of extreme imbalance.

  Implicationpartial

  Directly stated in the abstract but presented as background/context rather than a primary finding

  Verificationpartial

  partial

• The AstroConcepts corpus enables systematic study of extreme class imbalance in scientific domains.method 90%
  Evidencepartial

  By releasing this resource, we enable systematic study of extreme class imbalance in scientific domains and establish strong baselines across traditional, neural, and vocabulary-constrained LLM methods.

  Implicationpartial

  Directly stated as a purpose of the resource in the abstract

  Verificationpartial

  partial

Competitive landscape

A new corpus and evaluation framework for tackling extreme class imbalance in scientific text classification, enabling more robust NLP models for specialized domains.