Skip to content

From the Fluency Fallacy to the Micro-to-Macro Validity Gap: Opportunities and Pitfalls of LLMs in Social Simulation

Jul 2025 · Frontiers in Artificial Intelligence · 39 citations · 93 references
Computer Science

TL;DR

This critical review examines the use of LLMs as cognitive or decision-making components of simulated agents from a computational social science perspective, and argues that their behavioural and epistemic limitations can become systemic vulnerabilities when scaled to multi-agent societies.

Abstract

The integration of Large Language Models (LLMs) into social simulation has generated considerable enthusiasm, but also raises substantial methodological and epistemological challenges. This critical review examines the use of LLMs as cognitive or decision-making components of simulated agents from a computational social science perspective. Rather than treating the psychological evaluation of LLMs as separate from simulation mechanics, we argue that their behavioural and epistemic limitations can become systemic vulnerabilities when scaled to multi-agent societies. We first map the rapidly evolving landscape of LLM-driven platforms, ranging from small narrative sandboxes to population-scale and spatially structured simulations. We then develop a unified critical framework for analyzing the Micro-to-Macro Validity Gap: the propagation and amplification of micro-level limitations, including hallucinations, stochastic inconsistency, representational biases, and alignment effects, into macro-level risks such as the Fluency Fallacy, convergence toward an average persona, and automation bias. We identify contexts in which LLM-based agents offer genuine operational value, including serious games, participatory environments, and exploratory modelling, while distinguishing these uses from confirmatory research and precise social forecasting. Finally, we examine theory-driven hybrid architectures that embed LLMs within explicit, mechanistic, and reproducible Agent-Based Modelling (ABM) frameworks. We argue that such architectures offer a promising but not sufficient path toward improving epistemic control: their validity depends on multi-level evaluation of environmental dynamics, individual behaviour, cross-level interactions, and aggregate outcomes, and they remain vulnerable to the risk of physics washing.

View source

Similar papers

#artificial intelligence Preprint Nov 2026

Exploit or conform? How norm utility prompting changes the way LLMs use community beliefs

Large Language Models (LLMs) serve both as tools for aiding human decision-making in everyday life and as models of human agents in experimental simulation studies. Alignment on prototypical social decisions -- such as whether to cooperate on a collective action -- therefore has immediate practical consequences for res...

Leon Klingborg, Theodor Cimpeanu, Alexander J. Stewart · 0 citations
Open access Oct 2026

Need for Cognitive Closure and Ideological Rigidity: A Conditional Cross-Level Framework of Political Judgment

In this conceptual article, we explore possible relationships between the need for cognitive closure (NCC), uncertainty regulation, ideological rigidity, and the conditions of democratic deliberation. Kruglanski’s lay epistemic theory accounts for the motivational basis of closure, its dispositional and situational var...

Lokman Şahin, Cemile Burcu Kartal · 0 citations
Review Open access Nov 2026

A Survey on LLMs from a Human-Centric Perspective

The rapid evolution of large language models (LLMs) has positioned them as powerful computational instruments for human-centered research and applications, spanning tasks involving cognition, decision-making, and social interaction. This survey examines LLMs from a human-centric perspective organized around individual...

Jing-Yi Wang, Nicholas Sukiennik, Tong Li et al. · 0 citations
#small language model Open access Oct 2026

Performance within the foundation paradox: the case for small qualitative evaluations

This paper argues for SQEs as a complementary evaluation strategy for AI systems operating in dynamic, contested, and interdisciplinary settings and introduces small qualitative evaluations (SQEs) as a human-in-the-loop framework for assessing LLM performance in such less-bounded domains.

Michael P. Simeone, Jacki Hyatt, E. Bienenstock et al. · 0 citations
Conference Open access Oct 2026

The Artificial Moral Machine: Do LLMs Mirror Human Moral Diversity or Merely Caricature It?

Persona conditioning is the practice of prompting LLMs to adopt specific demographic identities. Although it is increasingly used to simulate human perspectives, its fidelity in the moral domain is untested at scale. We address this gap by testing moral fidelity on three LLMs of different sizes: gpt-4.1-nano (the small...

Davide Liga, Liu-Wen Yu, M. Slavkovik et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.