Skip to content
Review

Using AI Predictions to Augment Rather Than Replace Surveys

Nov 2026 · 0 citations
Computer Science

TL;DR

This study tests whether AI predictions can improve the statistical efficiency of social surveys when combined with census-scale auxiliary information and finds that supervised predictors yield effective-sample-size gains of about 7-12 percent when estimating the mean, while LLM gains are smaller but positive.

Abstract

LLMs can predict human survey responses with nontrivial accuracy. What this means for the future of social surveys remains hotly contested. Optimists anticipate a coming era of synthetic respondents; skeptics see a fragile and overhyped substitute. This article advocates a position: AI predictions cannot replace surveys, but they can augment them. Using prediction-powered inference (PPI), a statistical framework rooted in survey methodology and known in sociology as"mixed subjects design", the study tests whether AI predictions can improve the statistical efficiency of social surveys when combined with census-scale auxiliary information. Applying the framework to 83 social outcomes in the General Social Survey (GSS) and National Health Interview Survey (NHIS), with American Community Survey (ACS) being the auxiliary file, we find that supervised predictors yield effective-sample-size gains of about 7-12 percent when estimating the mean, while LLM gains are smaller but positive.

View source

Similar papers

Review Nov 2026

Necessary for What? Auditing Demographic Disclosure to LLM Synthetic Respondents

Large language models (LLMs) increasingly answer surveys on behalf of real panelists, and each panelist's recorded demographic attributes (e.g., age, education, ethnicity) are often placed in the prompt on the premise that richer profiles make more faithful synthetic respondents. While data minimization limits personal...

Aspen Wang, Yen Yu Shan, Lun-Wei Ku · 0 citations
Review Open access Sep 2026

Prediction-Based Adaptive Designs for Reducing Wave Nonresponse Rates and Bias in Panel Surveys

Machine learning (ML)-based nonresponse prediction in panel surveys enables selective interventions. However, the optimal use of ML predictions in Adaptive Survey Design (ASD) remains uncertain. We propose a method that integrates field experiment results on incentives, questionnaire length, and questionnaire content w...

John J. Collins, Saskia Bartholomäus, Tobias Gummer et al. · 0 citations
Review Nov 2026

Can Model Answer Probabilities Represent Survey Populations? Conditional Prediction and Joint Validity in Jev Survey Simulation

Survey simulations require realistic combinations of answers, but predictive accuracy alone does not establish that a model can generate them. We evaluate the option probabilities returned by Jev, a TypeSafe decision model, using linked responses to 20 binary items from five archived survey waves. We compare population...

Zeng-Qing Wu, Chuan Xiao · 0 citations
Book Open access Nov 2026

Predict Me if You Can: Simulating User Responses with Synthetic Data: Evaluating the Fidelity of LLM-Based Synthetic Respondents

Conditioning language models on persona profiles is standard practice for generating synthetic survey responses, yet the relative contribution of different attributes to the fidelity of synthetic survey responses has not been systematically quantified. To test this, we conditioned four large language models (LLMs) on p...

Seoyoung Kim, Seonju Seo, Joseph Nak-Hyung Song et al. · 0 citations
Preprint Oct 2026

Organizing LLM Inference for Aggregate Predictions from Behavioral Data

When using a large language model (LLM) to simulate a user population, how should we combine individual behavioral histories to predict the fraction of users who take a particular action? We study this question in e-commerce, predicting observed product-level conversion rates from browsing sessions without task-specifi...

Varun Gupta, Vijay Kamble · 0 citations
#small language model Preprint Oct 2026

COMPASS: Finding Where Reasoning Lives in Language Models

Comparing three model families and multiple math benchmarks, COMPASS outperforms the activation-steering baselines the authors compare against, improves GSM8K accuracy by 16 percentage points on average, and approaches CoT accuracy with 20-70\% fewer generated tokens.

Pratyay Dutta, Kowshik Thopalli, V. Narayanaswamy · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.