Designing Psychologically Safe AI Tutors for Students: An Emotion-Aware Post-Hoc Intervention for LLM-Assisted Learning

Abstract

Large language models (LLMs) are increasingly integrated into students' learning processes, serving not only as cognitive tools for academic support but also as conversational partners that influence learners' emotional experience. However, existing LLM-based learning systems are not explicitly designed to account for learners' emotional vulnerability, and institutions often deploy such models as black-box services, limiting the feasibility of model-level modification. In this work, we investigate the prevalence of emotionally risky responses in realworld LLM-assisted learning and propose a non-invasive, posthoc emotion-value gated framework that enhances emotional safety and supportive communication without altering underlying model architectures. Through a large-scale analysis of 42,172 authentic student-AI interactions and a controlled deployment involving 24 students, we show that emotionally risky responses occur with meaningful frequency in baseline usage and can be substantially reduced by our approach. Expert evaluations demonstrate significant improvements in emotional safety, supportiveness, and overall response quality, while student studies indicate increased comfort, engagement, and willingness to use the emotion-enhanced AI, with no perceived loss in response correctness. These findings highlight the importance of emotional governance in educational AI and demonstrate that system-level, deployable interventions can meaningfully improve students' learning experience in LLM-assisted environments.