Context-Aware Student Engagement Detection with MLLM

Abstract

Student engagement detection is an important component of adaptive learning systems, yet it remains challenging because observable behaviours can have different meanings depending on the context. Most existing approaches primarily infer engagement from visual cues, while other contextual information is less frequently incorporated. This work investigates whether a Multimodal Large Language Model can estimate student engagement through prompt-based reasoning without task-specific training. Using the CASED dataset, we propose a structured prompting architecture that jointly processes student and instructor key frames, a slide snapshot, and textualized facial and head-behaviour descriptors extracted with MediaPipe. The model is guided through a fixed question-answering schema that first characterizes the teaching context, then analyses the student’s observable behaviour, and finally integrates both sources of information to assess engagement. Although the engagement classification results are not fully satisfactory, our analysis suggests that vision-language models appear promising for this task.