Recognition of Affective States Using Multimodal and Adaptive AI for Individuals with Motor Disabilities

Abstract

People with motor disabilities often display atypical emotional expressions that elude conventional recognition models, leading to communication difficulties, isolation, and restricted access to emotional support. Multimodal AI affective recognition systems offer promising avenues to overcome these barriers, but existing approaches do not adapt to the expressive uniqueness of this population. This research aims to design, develop, and validate an affective-state recognition system for people with motor disabilities, using a multimodal and adaptive AI capable of learning each user’s unique "expressive emotional language". We identify three interrelated challenges: (1) identifying reliable individual affective behavioral markers without universal norms, while ensuring interpretability; (2) designing multimodal adaptive architectures that robustly fuse noisy or incomplete signals within lightweight, real-time constraints; and (3) validating the system in real-world settings while addressing ethical considerations such as bias, transparency, and privacy. For each challenge, we discuss candidate approaches worth exploring. Ultimately, this research aims to provide an operational prototype for affective communication assistance, supporting social inclusion and well-being of the individuals concerned, while advancing scientific knowledge in adaptive affective computing and inclusive AI.