Robin Welsch

but none the wiser: The disconnect between performance and metacognition

When the AI Leaves the Tailorshop: Measuring What an LLM Advisor Leaves Behind in Complex Problem Solving

Complex problem solving depends on acting effectively and understanding how a system works. AI advice may support these outcomes unequally. Two preregistered experiments compared participants managing a simulated clothing factory with and without an LLM advisor. Across studies, AI-supported participants reported greate...

Confident, Not Wiser: The Dunning-Kruger Effect in Human-AI Interaction

This study compares Human alone and Human+AI performance on reasoning tasks, for which the AI model is benchmarked on the same items, and distinguishes performance augmentation from metacognitive augmentation and motivate interfaces that support verification, communicate task-specific AI model performance, and help use...

How Did Writing Change At CHI? Analyzing 44 Years of CHI Writing Before and After the Introduction of Large Language Models

The availability of Large Language Models (LLMs) reshaped scientific discourse at a linguistic level. LLMs are assumed to homogenize academic writing, flattening it into a single generic lexical register. To understand how CHI writing has changed since the public release of LLMs, we analyzed full texts of 14,262 archiv...

Beyond"ChatGPT Can Make Mistakes": Designing Interventions to Support Metacognitive Monitoring in AI-Assisted Work

A shared vocabulary, a design space, and evidence that measured monitoring and task performance are separable design targets are contributed, suggesting that measured monitoring and task performance are separable design targets.

Available but Unclaimed: An Empirical Study of Human-AI Synergy

Evaluating LLMs in interaction with humans and designing support for selective deference that preserves independent reasoning motivate evaluating LLMs in interaction with humans.