Artificial Intelligence vs. Traditional Models in Computational Thinking Gap Analytics: A BERT-Driven Approach with XAI Diagnostics

Abstract

This study investigates computational thinking (CT) proficiency in programming-based learning environments using a unified analytical framework that combines statistical analysis and predictive modeling. CT proficiency is examined across five pedagogically grounded dimensions—abstraction, decomposition, algorithmic thinking, debugging, and pattern recognition—derived from rubric-based assessment of student work. First, descriptive and inferential statistical analyses are conducted to examine overall CT performance and gender-related patterns, providing correlational insight into group-level differences. Building on this analysis, a transformer-based model (BERT) is employed to predict continuous CT proficiency from students’ code comments and reflective journals, enabling semantic modeling of higher-order reasoning expressed in natural language. The predictive performance of BERT is benchmarked against traditional machine learning and lightweight deep learning baselines. Results show that transformer-based semantic modeling improves predictive accuracy while maintaining interpretability through post hoc explanation methods. Explainable AI techniques are used to identify linguistic and behavioral indicators associated with CT proficiency, and gender-related interpretations are derived through subsequent comparative analysis rather than direct prediction. Overall, the study positions deep learning as a complementary tool to statistical analysis for understanding and predicting CT proficiency.