Jul 2025· Frontiers in Artificial Intelligence· 39 citations· 93 references
Computer Science
TL;DR
This critical review examines the use of LLMs as cognitive or decision-making components of simulated agents from a computational social science perspective, and argues that their behavioural and epistemic limitations can become systemic vulnerabilities when scaled to multi-agent societies.
Abstract
The integration of Large Language Models (LLMs) into social simulation has generated considerable enthusiasm, but also raises substantial methodological and epistemological challenges. This critical review examines the use of LLMs as cognitive or decision-making components of simulated agents from a computational social science perspective. Rather than treating the psychological evaluation of LLMs as separate from simulation mechanics, we argue that their behavioural and epistemic limitations can become systemic vulnerabilities when scaled to multi-agent societies. We first map the rapidly evolving landscape of LLM-driven platforms, ranging from small narrative sandboxes to population-scale and spatially structured simulations. We then develop a unified critical framework for analyzing the Micro-to-Macro Validity Gap: the propagation and amplification of micro-level limitations, including hallucinations, stochastic inconsistency, representational biases, and alignment effects, into macro-level risks such as the Fluency Fallacy, convergence toward an average persona, and automation bias. We identify contexts in which LLM-based agents offer genuine operational value, including serious games, participatory environments, and exploratory modelling, while distinguishing these uses from confirmatory research and precise social forecasting. Finally, we examine theory-driven hybrid architectures that embed LLMs within explicit, mechanistic, and reproducible Agent-Based Modelling (ABM) frameworks. We argue that such architectures offer a promising but not sufficient path toward improving epistemic control: their validity depends on multi-level evaluation of environmental dynamics, individual behaviour, cross-level interactions, and aggregate outcomes, and they remain vulnerable to the risk of physics washing.
Large Language Models (LLMs) serve both as tools for aiding human decision-making in everyday life and as models of human agents in experimental simulation studies. Alignment on prototypical social decisions -- such as whether to cooperate on a collective action -- therefore has immediate practical consequences for res...
Leon Klingborg, Theodor Cimpeanu, Alexander J. Stewart· 0 citations
In this conceptual article, we explore possible relationships between the need for cognitive closure (NCC), uncertainty regulation, ideological rigidity, and the conditions of democratic deliberation. Kruglanski’s lay epistemic theory accounts for the motivational basis of closure, its dispositional and situational var...
Lokman Şahin, Cemile Burcu Kartal· The social science· 0 citations
The rapid evolution of large language models (LLMs) has positioned them as powerful computational instruments for human-centered research and applications, spanning tasks involving cognition, decision-making, and social interaction. This survey examines LLMs from a human-centric perspective organized around individual...
Jing-Yi Wang, Nicholas Sukiennik, Tong Li et al.· ACM Computing Surveys· 0 citations
This paper argues for SQEs as a complementary evaluation strategy for AI systems operating in dynamic, contested, and interdisciplinary settings and introduces small qualitative evaluations (SQEs) as a human-in-the-loop framework for assessing LLM performance in such less-bounded domains.
Michael P. Simeone, Jacki Hyatt, E. Bienenstock et al.· Neural computing & applicati...· 0 citations
Persona conditioning is the practice of prompting LLMs to
adopt specific demographic identities. Although it is
increasingly used to simulate human perspectives, its
fidelity in the moral domain is untested at scale. We
address this gap by testing moral fidelity on three LLMs of
different sizes: gpt-4.1-nano (the small...
Davide Liga, Liu-Wen Yu, M. Slavkovik et al.· Proceedings of the AAAI/ACM...· 0 citations