Does Topic Sentiment Cause Perceived Ideology? Comparing Human and LLM Annotations in Political News Articles

Dimensions overall score 4.0

• Fine-tuned LLMs exhibit spurious sentiment-ideology couplings invisible to standard evaluation, impacting their use as proxies for human judgment. Using articles from AllSides, paired with shared sentiment annotations from Llama-3.3-70b-versatile, we compare ideology labels from expert human annotators, GPT-4o-mini (baseline and finetuned), and Llama-3.3-70B.problem 35%
  Evidencepartial

  Fine-tuned LLMs exhibit spurious sentiment-ideology couplings invisible to standard evaluation, impacting their use as proxies for human judgment. Using articles from AllSides, paired with shared sentiment annotations from Llama-3.3-70b-versatile, we compare ideology labels from expert human annotators, GPT-4o-mini (baseline and finetuned), and Llama-3.3-70B.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• We ask whether topic sentiment has a causal effect on perceived political ideology, and whether the answer depends on who assigns the ideology label. Using articles from AllSides, paired with shared sentiment annotations from Llama-3.3-70b-versatile, we compare ideology labels from expert human annotators, GPT-4o-mini (baseline and finetuned), and Llama-3.3-70B.method 35%
  Evidencepartial

  We ask whether topic sentiment has a causal effect on perceived political ideology, and whether the answer depends on who assigns the ideology label. Using articles from AllSides, paired with shared sentiment annotations from Llama-3.3-70b-versatile, we compare ideology labels from expert human annotators, GPT-4o-mini (baseline and finetuned), and Llama-3.3-70B.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• ScienceToStartup currently rates this 4.0/10 on the public viability pass. Fine-tuned GPT-4o-mini achieves the highest classification accuracy (F1=72.48) and is the only annotator paradigm that produces significant community-level treatment effects and significant natural direct effects (NDEs) in mediation. A public repository is linked, so build verification can inspect implementation evidence instead of treating the paper as PDF-only.result 35%
  Evidencepartial

  ScienceToStartup currently rates this 4.0/10 on the public viability pass. Fine-tuned GPT-4o-mini achieves the highest classification accuracy (F1=72.48) and is the only annotator paradigm that produces significant community-level treatment effects and significant natural direct effects (NDEs) in mediation. A public repository is linked, so build verification can inspect implementation evidence instead of treating the paper as PDF-only.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

• LLM Evaluation moved forward this cycle; last verified June 2026. Public score 4.0/10. Implementation evidence is present through a linked repository.why now 35%
  Evidencepartial

  LLM Evaluation moved forward this cycle; last verified June 2026. Public score 4.0/10. Implementation evidence is present through a linked repository.

  Implicationpartial

  Abstract-backed fallback claim; anchored extraction has not materialized a public claim row yet.

  Verificationpartial

  partial

Competitive landscape

Fine-tuned LLMs exhibit spurious sentiment-ideology couplings invisible to standard evaluation, impacting their use as proxies for human judgment.