Automated Misconception Diagnosis & Pedagogical Remediation: Using Generative AI to Identify and Remedy Structural Bugs in Introductory Code
Abstract
Novice programmers in introductory computer science (CS1) courses frequently experience cognitive overload, helplessness, and attrition due to structural code bugs. Unlike superficial syntax errors caught by standard compilers, structural bugs stem from fundamental misalignments between a student's mental model and the runtime execution context. While recent advances in Generative Artificial Intelligence (GenAI) and Large Language Models (LLMs) demonstrate remarkable code generation capabilities, generic LLM feedback often defaults to providing direct, answer-giving code patches. This passive resolution deprives students of essential problem-solving opportunities and exacerbates misconception persistence. This paper introduces StructDiag-AI, a novel novel Socratic diagnostic framework that integrates Abstract Syntax Tree (AST) static analysis with constrained LLM prompt synthesis. By isolating AST structural divergences prior to prompt assembly, StructDiag-AI identifies non-viable algorithmic patterns (e.g., misnested loops, unreachable recursive branches, and state mutation outside iteration) and generates progressive, non-generative Socratic pedagogical hints. We present a multi-institutional empirical trial ($N = 482$ CS1 students) evaluating StructDiag-AI against traditional compiler errors and baseline unconstrained LLM feedback. Quantitative results demonstrate that StructDiag-AI increases novice bug resolution rates by 38.4% compared to standard compilers and improves long-term conceptual retention by 24.1% over unconstrained LLM feedback. Furthermore, qualitative analysis reveals a significant reduction in student frustration metrics and rapid convergence toward correct mental models.