Skip to content
Preprint

Diagnosing the Explicit-to-Applied Theory of Mind Gap in LLMs

Oct 2026 · 0 citations · 29 references
Computer Science

Abstract

As large language models are deployed in high-stakes roles such as therapy, education, and personal assistance, their social cognition must be assessed reliably. Current Theory of Mind (ToM) benchmarks report aggregate scores that conflate two distinct competencies: explicit ToM (recognizing what someone believes) and applied ToM (using that belief to predict behavior). Building on this distinction, we investigate the gap and its recoverability using four structurally diverse benchmarks. Aggregate reporting obscures uneven competence: on SimpleToM, Llama-3.1-8B scores 51.5 percent overall while recognizing mental states at 99.4 percent and predicting behavior at only 28.9 percent. To assess recovery, we compare structured prompting, matched in-context learning, and our proposed Belief-State Test-Time Training (BSTTT), which adapts lightweight parameters on answered support questions through action reconstruction. Across Qwen2.5-7B, Llama-3.1-8B, and GPT-oss-20B, recovery depends on the model, task, and intervention. On SimpleToM, reminders combined with BSTTT improve Llama's behavior prediction by 40.8 percentage points; reminders account for most of the overall improvement, with BSTTT adding 1.3 to 5.5 points beyond reminders across models. Contextual examples often outperform weight updates, while BSTTT remains stronger in specific model-benchmark settings. Temporal belief tracking remains difficult despite partial recovery. These findings show that aggregate ToM scores can overstate applied competence while hiding performance recoverable through targeted assistance. We recommend decomposed reporting of explicit and applied accuracy alongside intervention comparisons that establish which failures are recoverable and under what conditions.

View source

Similar papers

Training LLMs to Verbalize Evaluation Awareness

Verbalization training is introduced, a method for making LLMs less reticent about verbalizing evaluation awareness while avoiding to supervise the latent belief itself, and it is shown that VT-induced verbalizations reflect the richer knowledge acquired by the model.

Usman Anwar, Sahar Abdelnabi, D. Krueger · 0 citations
Review Oct 2026

PsycheMME: A Psychology-Informed Benchmark for Social Understanding and Interaction in Videos

Understanding human needs and selecting responses require distinct judgments that aggregate accuracy can obscure. We introduce PsycheMME, a video benchmark informed by psychology, with eight capabilities spanning grounding, states, needs, interaction decisions, memory, and risk. Drawing on Social Information Processing...

He-Ran Hu, Yue Dong, Zhu-Cun Xue et al. · 0 citations
Open access 2026

What Do You Mean? Exploring the Alleged Theory of Mind Abilities of Large Language Models

The results reveal that, although LLMs occasionally succeed in decoding communicative intentions, their performance is not attributable to human-like ToM reasoning, and offers insight into their interpretive biases, contributing to a deeper understanding of their linguistic capabilities.

Agnese Lombardi, Alessandro Lenci · 0 citations
#natural language process... Preprint Oct 2026

MindEval2: Clinically Grounded Patients and Rubrics for Evaluating Language Models in Mental Health Support

With people increasingly turning to AI chatbots for mental health support, the need to evaluate these tools is becoming more pressing. Recent benchmarks attempt to mimic a realistic setting by simulating multi-turn interactions with synthetic patients and evaluating them with an LLM-as-a-judge. However, the usefulness...

John Mendonça, Daniel Tschernutter, José Pombal et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Collaborative Theory of Mind: Learning to Self-Revise by Anticipating User Responses

Large language models (LLMs) demonstrate strong individual task performance, yet still struggle to collaborate with users over extended, multi-turn interactions, for example by attempting solutions before understanding the user's needs or by failing to revise mistaken assumptions after receiving feedback. We propose Co...

Fan-Zhi Zeng, Anubrata Das, M. Saar-Tsechansky et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.