Teaching Programming in the Age of Generative Artificial Intelligence: Learning Gains and Pedagogical Integration in a Higher Education Context

Abstract

The rapid integration of Generative Artificial Intelligence (GenAI) into programming education has raised important questions regarding its impact on learning processes, conceptual understanding, and technological dependency. This study analyzed the effects of four GenAI-supported instructional strategies in an introductory programming course for undergraduate engineering students. A multi-group quasi-experimental pre-test–post-test design was implemented involving 686 students distributed across 53 class groups, from 10 campuses, taught by 32 professors. The instructional conditions included Quizzes for Self-Regulation, Github-Copilot-assisted learning, Prompt Problems with Iterative Refinement, and Flipped Learning enhanced with GenAI, which were compared against a traditional teaching approach. Learning outcomes were measured using normalized learning gain, while statistical analyses were conducted using non-parametric methods due to deviations from normality and heteroscedasticity. Results indicate that GenAI integration did not produce statistically significant overall differences in learning gain when all GenAI-supported strategies were analyzed as a single cluster compared to traditional instruction. However, differences emerged between specific strategies, with Quizzes and Copilot-based approaches having higher median learning gains than Prompt Problems and Flipped Learning strategies. No statistically significant differences associated with gender were identified. These findings suggest that the effectiveness of GenAI in programming education depends less on the mere presence of the technology and more on the pedagogical conditions under which it is integrated into the teaching–learning process.