Skip to content
Book Open access

Predict Me if You Can: Simulating User Responses with Synthetic Data: Evaluating the Fidelity of LLM-Based Synthetic Respondents

Nov 2026 · 0 citations

Abstract

Conditioning language models on persona profiles is standard practice for generating synthetic survey responses, yet the relative contribution of different attributes to the fidelity of synthetic survey responses has not been systematically quantified. To test this, we conditioned four large language models (LLMs) on profiles of 258 employed South Korean adults surveyed on AI-related job replacement risk. We grouped 11 attributes into three blocks (demographics, socioeconomic position & ideology, experience) and evaluated four cumulative levels of persona richness. A leave-one-out ablation identified the most impactful block. Fidelity generally rose with richer personas, and ablation showed that only experience-related attributes contributed meaningfully, while demographic attributes were net-neutral at best. For human-agent interaction (HAI) research with synthetic respondents, fidelity hinges on attribute relevance, not information quantity — neither the largest model nor all attributes maximize it.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.