Towards a Scenario-Based Framework for Developing Multiple Learning Analytics Dashboards
TL;DR
An intervention involving teachers and school principals was conducted to illustrate the operationalization of the first two scenarios and provide initial evidence of their feasibility in authentic educational settings, and the framework was perceived as beneficial for understanding the various steps involved in creating MLADs.
Abstract
The significant amount of data available about students and their learning activities is spread across many digital educational systems today, making it challenging to use them effectively. This study addresses this issue by proposing a scenario-based framework for the design and development of Multiple Learning Analytics Dashboards (MLADs). The framework guides researchers, developers, and educational stakeholders in designing MLADs that integrate educational data from multiple sources to support the decision-making needs of teachers, school principals, municipal education leaders, and Educational Technology (EdTech) companies across the micro, meso, and macro levels. In this study, the MLADs refer to having different Learning Analytics Dashboards (LADs) tailored for different stakeholders, with each LAD containing multiple pages. The framework operationalizes this process through four stakeholder-specific scenarios, each comprising eight steps that guide the collaborative design and development of stakeholder-specific LADs within an MLAD ecosystem. The proposed scenarios illustrate how data is handled, transferred, analyzed, and visualized to these stakeholders. To explore the application of the framework at the micro and meso levels, we conducted an intervention involving teachers and school principals to illustrate the operationalization of the first two scenarios and provide initial evidence of their feasibility in authentic educational settings. The framework was perceived as beneficial for understanding the various steps involved in creating MLADs and highlighted the significance of user involvement in the design process. These benefits include enhanced clarity and interpretation of educational data, support for understanding students’ learning patterns based on available outcomes and activities, and the potential to streamline routine tasks within the teacher scenario. They also encompass the capacity to compare classroom-level indicators across teachers and to interpret school-level patterns in the principal scenario.