Skip to content

Towards Detecting AI-Assisted Responses in Online Surveys

Sep 2026 · 0 citations · 29 references
Computer Science

TL;DR

An initial benchmark dataset is introduced, namely ASURRE, for AI-assisted survey participation to capture usage strategies ranging from full generation and revision to persona-grounded agentic completion, and it is shown that agentic completion cannot fully replicate respondent-level behaviour and leaves distinctive behavioural traces.

Abstract

The use of LLMs to complete online surveys impacts the validity of survey-based research, but detecting such usage remains underexplored. We introduce an initial benchmark dataset, namely ASURRE, for AI-assisted survey participation to capture usage strategies ranging from full generation and revision to persona-grounded agentic completion. Controlled by these strategies, LLM-assisted survey responses are generated using multiple LLMs on three real-world surveys in different disciplines, paired with genuine human responses. Our evaluation of existing machine-generated text (MGT) detectors shows that naive AI usage is readily detectable, whereas persona-grounded agents that mimic entire respondents push detector performance toward chance. We further show that agentic completion cannot fully replicate respondent-level behaviour and leaves distinctive behavioural traces. While individual cues can be circumvented by targeted prompting, a simple few-shot, training-free aggregator over these cues improves mean AUROC by +0.14 over the best existing detector across agentic settings. Our project is available at https://github.com/mike-qz-wang/ASURRE.

View source

Similar papers

AI-Assisted Submissions in Online Research Are Rare and Highly Concentrated but Routinely Approved

The results do not support the view that AI assistance currently poses an existential threat to online research, but reveal limited payment consequences for prohibited use and gaps in platform safeguards, leaving platforms poorly prepared should that threat materialize.

N. K. R. Sehgal, Manuel Tonneau, D. Folk et al. · 0 citations
#artificial intelligence Preprint Aug 2026

BLOOM-WILT: Logit Tilting for Behaviour Elicitation in Automated LLM Auditing

BLOOM-WILT is introduced, a full auditing pipeline that elicits natural multi-turn instances of rare behaviours, without training cost or access beyond the target's next-token distribution and raises average behaviour presence from 51% to 100% when eliciting self-harm encouragement from Qwen3.5-4B.

Adrians Skapars, Edoardo Manino · 0 citations

SCBO: Semantically Coherent Batching and Ordering for LLM-Based Social Surveys

Large Language Models (LLMs) offer a scalable way to simulate survey respondents using demographic profiles and observed reference responses. However, the conventional approach of predicting one question per prompt repeatedly encodes the same context, limits each target to a narrow set of reference responses, and preve...

Yuan-Zi Li, Ling-Jie Wang, Zi-Hang Tian et al. · 0 citations
Review Aug 2026

Hybrid Panels: Toward Human-AI Collaboration in Survey Research

The concept of a hybrid panel is introduced by providing a definition and outlining an overarching framework, spanning data collection to data validation, spanning data collection to data validation.

Julia Romberg, Tobias Gummer, Gabriella Lapesa et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.