A Metacognitive Blind Spot: Student Comprehension, AI Reliance, and the Conceptual Difficulty Gap

Abstract

This exploratory study investigates the relationship between student metacognition, use of artificial intelligence, and empirical performance within a multidisciplinary course on AI. Using data from a sample of college-age, full-time undergraduate students (averaging 18 participants per assessment) enrolled in an in-person junior seminar at a Midwestern U.S. university, we correlate student self-assessments with standard readability metrics (e.g., Flesch–Kincaid), L2SCA metrics, and objective assessment outcomes, analyzing how learners evaluate their own comprehension and how they deploy AI tools in response to the complexity of 21 reading assignments over 8 weeks. We find that students’ perceptions of linguistic difficulty correlate with classical readability scores, but their perceptions do not predict their success nor does their engagement with assistive AI. The results suggest that students utilize generative AI tools as a habitual baseline rather than a strategic response to difficult material. We argue that while students can identify surface-level linguistic friction, they fail to recognize deep conceptual hurdles, leading to a false sense of mastery that neither their intuition nor their AI assistants appear to mitigate. We propose that a quantifiable metric, the Conceptual Difficulty Gap (CDG), may be useful for identifying a class of texts that syntactically appear to be simple, but consistently trigger performance failures. Crucially, we uncover a possible metacognitive blind spot: student self-ratings of difficulty are negatively correlated with this gap, implying that student assessments of difficulty are not based on actual conceptual difficulty. Furthermore, self-reported AI reliance shows no correlation with the gap, indicating that students may not be strategically deploying generative AI tools to mitigate conceptual difficulty.