Think Before You Accept: Can Written Justification Reduce Uncritical Uptake of AI Writing Suggestions?
TL;DR
This work examines whether requiring students to justify decisions to accept or reject AI suggestions can mitigate uncritical uptake in academic writing and reveals superficial engagement in the justification task and gaps in metacognitive monitoring and domain knowledge.
Abstract
Generative AI can offer students useful feedback, but its value depends on judging which suggestions are accurate and relevant. Prior research shows that strategic friction during human-AI interactions can promote critical uptake, but how to effectively implement such friction in academic contexts remains unclear. We examine whether requiring students to justify decisions to accept or reject AI suggestions can mitigate uncritical uptake in academic writing. In a randomized experiment embedded in a course activity (N=129), students wrote a data analysis proposal, received mixed-quality AI revision suggestions, and decided whether to accept or reject them. Students required to provide written justifications were 24 percentage points less likely to adopt flawed suggestions (65% vs. 41%), with no reduction in acceptance of sound suggestions (81% vs. 86%). However, thematic analysis revealed superficial engagement in the justification task and gaps in metacognitive monitoring and domain knowledge.
