Video Data Regression Method for Physiological and Behavioral Cues Assessment Aimed to Contactless Mental Fatigue Classification

Abstract

Mental fatigue of computer operators poses significant risks to productivity, safety, and well-being, yet existing monitoring solutions remain either intrusive (relying on wearable sensors) or subjective (based on self-reports). To address this challenge we propose video data regression method for physiological and behavioral cues assessment aimed to human mental fatigue classification. The method is fully contactless, it uses standard human facial video. The proposed method integrates video-based models for estimating blood pressure and oxygen saturation with temporal RGB signal dynamics and behavioral features describing eye, mouth, and head movements extracted using Google MediaPipe. To overcome the lack of objective ground truth, a previously validated sensor-trained model is employed to generate minute-by-minute fatigue labels that reflect the same underlying cognitive processes as wearable physiological measurements. Class imbalance is addressed through multiple balancing strategies, and several machine learning and deep learning models are systematically evaluated, with categorical boosting (CatBoost) using class weighting achieving the most robust performance. Feature importance analysis further identifies a compact yet highly informative feature subset, improving computational efficiency without sacrificing accuracy. Evaluated using leave-one-subject-out cross-validation on prolonged operator sessions, the proposed framework demonstrates robust subject-independent generalization, highlighting the potential of video-derived physiological regression combined with behavioral analysis as a scalable, nonintrusive solution for continuous mental fatigue assessment in real-world occupational environments.