When more technology does not mean better learning: digital tool use, instructional quality, and mathematics achievement in everyday classroom settings

Abstract

Meta-analyses highlight that digital tool use can positively affect students’ mathematics achievement. However, these findings mainly derive from controlled studies with well-orchestrated technology integration, whereas evidence from regular classrooms remains scarce—particularly regarding how often digital tools are used in everyday settings and how this relates to students’ mathematics achievement. At the same time, instructional quality—encompassing basic dimensions such as classroom management, cognitive activation, and student support—is widely acknowledged as a key determinant of learning outcomes, but it has rarely been examined alongside digital tool use within the same study. To address these gaps, we examined both the frequency of digital tool use and instructional quality as predictors of mathematics achievement in everyday classroom settings, and additionally explored their association at the classroom level. In an exploratory extension, we investigated whether the relationship between digital tool use frequency and mathematics achievement varies across levels of instructional quality, implying a potential moderating role. To this end, we tested interaction effects between digital tool use frequency and each of the three basic dimensions. We applied a multilevel analysis of longitudinal data from 1,627 eighth-grade students across 72 classes from 30 German secondary schools. Mathematics achievement was assessed through curriculum-based pre- and post-tests and analyzed in relation to student-reported digital tool use frequency and student-reported instructional quality at both the student and class levels. Contrary to meta-analytic findings, we identified a small but statistically significant negative association between everyday digital tool use frequency and mathematics achievement. In line with prior research, instructional quality was positively associated with achievement. At the class level, the exploratory analyses provided no evidence that instructional quality moderates the relationship between digital tool use frequency and mathematics achievement. At the student level, however, a significant interaction effect between classroom management and digital tool use frequency occurred, indicating that the negative association between frequent digital tool use and achievement was weaker when classroom management was perceived as strong. To gain a more comprehensive understanding of these relationships, future research should develop more refined and integrated models of digital tool use and instructional quality.