Selective Deficits in LLM Mental Self-Modeling in a Behavior-Based Test of Theory of Mind

Dimensions overall score 7.0

• LLMs released before mid-2025 fail at all tested Theory of Mind tasks.result 90%
  Evidencepartial

  LLMs released before mid-2025 fail at all of our tasks

  Implicationpartial

  This is explicitly stated in the abstract and supported by the findings presented in the figures and text.

  Verificationpartial

  partial

• More recent LLMs achieve human-level performance on modeling the cognitive states of others.result 90%
  Evidencepartial

  more recent LLMs achieve human-level performance on modeling the cognitive states of others

  Implicationpartial

  This is explicitly stated in the abstract and supported by the text indicating an upward trend for 'other-modeling' tasks with recent LLMs.

  Verificationpartial

  partial

• Frontier LLMs fail at self-modeling tasks unless provided with a scratchpad for reasoning.result 90%
  Evidencepartial

  even frontier LLMs fail at our self-modeling task - unless afforded a scratchpad in the form of a reasoning trace

  Implicationpartial

  This is explicitly stated in the abstract and supported by the text contrasting performance with and without a scratchpad.

  Verificationpartial

  partial

• LLMs may use limited-capacity working memory for mental representations during a single forward pass, suggested by cognitive load effects on other-modeling tasks.technical 80%
  Evidencepartial

  we further demonstrate cognitive load effects on other-modeling tasks, offering suggestive evidence that LLMs are using something akin to limited-capacity working memory to hold these mental representations in mind during a single forward pass

  Implicationpartial

  The abstract suggests this based on observed cognitive load effects, indicating suggestive evidence rather than a definitive conclusion.

  Verificationpartial

  partial

• Reasoning models readily engage in strategic deception.result 85%
  Evidencepartial

  we show that they readily engage in strategic deception

  Implicationpartial

  The abstract states this as a finding from exploring the mechanisms by which reasoning models succeed.

  Verificationpartial

  partial

• A novel behavior-based test of Theory of Mind was developed requiring strategic action based on mental state representations.method 90%
  Evidencepartial

  We therefore develop a novel experimental paradigm that requires that subjects form representations of the mental states of themselves and others and act on them strategically rather than merely describe them

  Implicationpartial

  The abstract clearly describes the development of a new paradigm with specific requirements.

  Verificationpartial

  partial

• Human subjects were included in the study alongside LLMs to provide a baseline for performance.method 90%
  Evidencepartial

  We test a wide range of leading open and closed source LLMs released since 2024, as well as human subjects, on this paradigm

  Implicationpartial

  The abstract explicitly mentions testing human subjects alongside LLMs.

  Verificationpartial

  partial

• LLMs released before mid-2025 fail at all of the developed Theory of Mind tasks.result 90%
  Evidencepartial

  We find that 1) LLMs released before mid-2025 fail at all of our tasks

  Implicationpartial

  This is explicitly stated in the abstract and supported by the findings presented in Figure 2 (nonthinking models).

  Verificationpartial

  partial

• More recent LLMs achieve human-level performance on modeling the cognitive states of others (other-modeling tasks).result 90%
  Evidencepartial

  2) more recent LLMs achieve human-level performance on modeling the cognitive states of others

  Implicationpartial

  This is explicitly stated in the abstract and supported by the trend shown in Figure 2 (nonthinking models) for other-modeling tasks.

  Verificationpartial

  partial

• Even frontier LLMs fail at self-modeling tasks unless provided with a scratchpad for reasoning traces.result 90%
  Evidencepartial

  3) and even frontier LLMs fail at our self-modeling task - unless afforded a scratchpad in the form of a reasoning trace.

  Implicationpartial

  This is explicitly stated in the abstract and supported by the comparison between 'nonthinking' and 'thinking' models in Figure 3.

  Verificationpartial

  partial

• LLMs may utilize limited-capacity working memory for mental representations during a single forward pass, as evidenced by cognitive load effects on other-modeling tasks.technical 80%
  Evidencepartial

  We further demonstrate cognitive load effects on other-modeling tasks, offering suggestive evidence that LLMs are using something akin to limited-capacity working memory to hold these mental representations in mind during a single forward pass.

  Implicationpartial

  The abstract suggests this as 'suggestive evidence' based on observed cognitive load effects.

  Verificationpartial

  partial

• Reasoning models (LLMs with scratchpads) readily engage in strategic deception.result 85%
  Evidencepartial

  Finally, we explore the mechanisms by which reasoning models succeed at the self- and other-modeling tasks, and show that they readily engage in strategic deception.

  Implicationpartial

  This is stated in the abstract as a finding from exploring the mechanisms of successful self- and other-modeling.

  Verificationpartial

  partial

Competitive landscape

This research develops a novel test for LLM Theory of Mind, revealing that while recent models excel at understanding others, they struggle with self-modeling unless given a scratchpad, suggesting a path to more sophisticated AI reasoning.