A Fairness-Aware Multimodal Deep Learning Architecture for Facial Aesthetic Evaluation Using Image and Metadata Fusion

Abstract

This investigation aimed to address the pressing concern about demographic bias inherent in artificial intelligence systems designed for facial aesthetic assessment. Although AI frameworks demonstrate considerable capability in analyzing facial morphology for cosmetic applications, evaluation metrics exhibit systematic variation across racial and gender categories. This study introduces a Fairness-Aware Multimodal AI architecture that fuses facial images with demographic metadata, incorporating explicit fairness constraints embedded within the training process. The proposed system utilizes ResNet50-based feature extraction within a multi-task learning paradigm that simultaneously optimizes predictive performance alongside equitable representation. A fairness-aware loss function minimizes performance differences across designated racial groups, specifically East Asian, Southeast Asian, Indian, Black, White, Middle Eastern, and Latino/Hispanic, as well as gender classifications. Experimental findings demonstrate that the proposed architecture achieves strong predictive performance (MSE 0.0301, MAE 0.1310 on a 0–1 scale) while significantly minimizing the demographic parity gap (approaching 0.012) without suffering from model collapse and preserving discriminative capacity across groups. A web-based interface built on FastAPI backend infrastructure enables users to conduct real-time evaluations. The proposed framework optimally balances predictive accuracy with ethical AI principles, delivering objective and unbiased aesthetic assessment suitable for cosmetic and visual analysis applications.