Evaluating the quality of learning design with learning analytics

Abstract

This study aims to support quality assurance in education by exploring and clarifying the concept of learning design quality (LDQ) and how learning analytics (LA), specifically design analytics, can be used to enhance it. The authors first provide structure to the concept of LDQ by conducting a rapid scoping review of relevant literature. Then the authors conduct an empirical evaluation of LDQ by analysing a large sample of 184 course learning designs (LDs), including 12,096 teaching and learning activities, developed in the innovative balanced design planning LD tool. This study revealed two broad dimensions of LDQ which can be explored through LA: aspects that can be determined a priori (before implementation) and a posteriori (during/after implementation), with community feedback related to both. While a priori evaluation draws on design analytics and content checks, a posteriori evaluation relies on LA and academic analytics to assess implementation. Empirical findings from a priori evaluation suggest educators should pay particular attention to planning in accordance with contemporary pedagogies. Besides providing the needed conceptual clarity regarding LDQ and using a large sample of LDs to demonstrate a priori evaluation, this study proposes quality checks in two cycles (during design and during/after implementation), which can be used to evaluate whether LD meets the criteria for a quality mark.