An initial benchmark dataset is introduced, namely ASURRE, for AI-assisted survey participation to capture usage strategies ranging from full generation and revision to persona-grounded agentic completion, and it is shown that agentic completion cannot fully replicate respondent-level behaviour and leaves distinctive behavioural traces.
Abstract
The use of LLMs to complete online surveys impacts the validity of survey-based research, but detecting such usage remains underexplored. We introduce an initial benchmark dataset, namely ASURRE, for AI-assisted survey participation to capture usage strategies ranging from full generation and revision to persona-grounded agentic completion. Controlled by these strategies, LLM-assisted survey responses are generated using multiple LLMs on three real-world surveys in different disciplines, paired with genuine human responses. Our evaluation of existing machine-generated text (MGT) detectors shows that naive AI usage is readily detectable, whereas persona-grounded agents that mimic entire respondents push detector performance toward chance. We further show that agentic completion cannot fully replicate respondent-level behaviour and leaves distinctive behavioural traces. While individual cues can be circumvented by targeted prompting, a simple few-shot, training-free aggregator over these cues improves mean AUROC by +0.14 over the best existing detector across agentic settings. Our project is available at https://github.com/mike-qz-wang/ASURRE.
It is found that students'detection accuracy improves over time, driven by a shift from relying on linguistic cues to leveraging shared social and contextual signals.
Dan Schumacher, Pragathi Durga Rajarajan, Haven Kotara et al.· 0 citations
The results do not support the view that AI assistance currently poses an existential threat to online research, but reveal limited payment consequences for prohibited use and gaps in platform safeguards, leaving platforms poorly prepared should that threat materialize.
N. K. R. Sehgal, Manuel Tonneau, D. Folk et al.· 0 citations
BLOOM-WILT is introduced, a full auditing pipeline that elicits natural multi-turn instances of rare behaviours, without training cost or access beyond the target's next-token distribution and raises average behaviour presence from 51% to 100% when eliciting self-harm encouragement from Qwen3.5-4B.
Large Language Models (LLMs) offer a scalable way to simulate survey respondents using demographic profiles and observed reference responses. However, the conventional approach of predicting one question per prompt repeatedly encodes the same context, limits each target to a narrow set of reference responses, and preve...
Yuan-Zi Li, Ling-Jie Wang, Zi-Hang Tian et al.· 0 citations
This work introduces a minimal "observer-effect" prompting technique, which demonstrates that subtle social-facilitation cues can reliably activate higher vigilance in LLMs without the need for additional reasoning instructions.
The concept of a hybrid panel is introduced by providing a definition and outlining an overarching framework, spanning data collection to data validation, spanning data collection to data validation.
Julia Romberg, Tobias Gummer, Gabriella Lapesa et al.· arXiv.org· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 20, 2026
Senior Research Scientist Jim Glass discusses how his MIT-IBM Computing Research Lab collaboration connects perception and language, and addresses the shortcoming of current models.
MIT News · Artificial Intelligence· news.mit.eduNov 19, 2026
An MIT and Dana-Farber Institute-led team used large language models to examine nearly 4,000 studies and hundreds of code repositories, revealing gaps in the information needed to scrutinize and reuse published research.
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.