“It Works, but I Don’t Know Why”: Uncritical Use, Verification Behavior and Perceived Understanding in Non-CS Students’ Generative AI-Assisted Programming Projects

Abstract

Generative artificial intelligence (GenAI) enables students with little programming experience to produce working software, raising the concern that learners may build artifacts they cannot explain. This convergent mixed-methods study examined how uncritical GenAI use and verification behavior (VB) relate to novices’ self-reported understanding of their own AI-assisted projects. Participants were undergraduates from predominantly non-computing majors in two offerings of a general-education Python course; 64 provided complete survey data and 68 written reflections were thematically analyzed. Uncritical use (UR) was associated with a larger self-reported product–understanding gap (GAP; r = 0.43, p < 0.001) but showed little association with perceived understanding (PU), whereas VB was associated with PU (r = 0.45, p < 0.001) but showed little association with the gap. Tests of dependent correlations confirmed that these associations differed. In the reflections, execution-based checking was the most commonly reported verification strategy (47 of 68), whereas comprehension-oriented strategies were less common. Because all measures were self-reported and cross-sectional, the findings are correlational; they suggest that instruction should explicitly scaffold comprehension-oriented verification in AI-integrated introductory programming.