How Model Choice and Memory Shape Preference Consistency in Large Language Models

Abstract

When a language model answers a sequence of questions, researchers report the model and temperature but rarely what or how it was told about its own earlier answers. I show that this underreported setting behaves as a mode effect. Testing four large language model (LLM) families with revealed preference theory (GARP) across moral, economic, and social tasks ( N = 5,926 ), I find that how prior choices re-enter the prompt, the memory condition , moves the share of internally consistent subjects by up to 48 percentage points, mostly where that share starts lowest. Memory also influences which preferences the models express and, in a silicon-sampling extension, it brings the answers closer to the human distribution. Nonetheless, gains in consistency and surface fidelity do not necessarily translate into representativeness: a consistent, human-looking model may not represent any human population. For LLMs as social-science instruments, this study offers promise, caution, and a method for testing the stability of sequential outputs.