ReflectAI: An AI-Driven Multimodal Journaling App for Promoting Mental Well-being
Abstract
Emotional well-being is fundamental to mental health, yet many individuals find it difficult to recognize and manage their emotional states in daily life. In 2022, the World Health Organization (WHO) reported that more than 1 billion people are currently living with emotional disorders, with conditions such as anxiety and depression placing a significant burden on both individuals and society. Current digital emotional wellness tools rely on manual self-report mechanisms and offer predefined emotion options for users to choose from, which limit their capacity to support personalized interventions and influence mood change. To address these limitations, this study presents ReflectAI, a mobile application that uses multimodal emotion detection to accurately capture the emotional state of the user and deliver personalized interventions for emotional well-being. ReflectAI captures the emotional state of the user through three input modalities: typed text, voice recordings, and facial images. Each input is then processed by a dedicated artificial intelligence model to predict the emotional state of the user along with a confidence score. Based on the emotions detected, the system delivers four personalized interventions, namely motivational quotes, music recommendations, meditation videos, and breathing exercises, to support mood regulation. To evaluate ReflectAI, an expert evaluation was conducted with 5 Human-Computer Interaction (HCI) experts using 10 heuristics adapted for persuasive health technologies. The results show an overall mean severity of 0.32 (SD = 0.47), with no rating exceeding the cosmetic level, indicating little or no usability problems across the system. The findings are used to inform a set of practical design guidelines for researchers who seek to develop future emotional awareness applications that integrate multimodal emotion detection with personalized intervention delivery to promote emotional well-being.