Artificial Intelligence-Enabled Medical Devices (AIeMD) promise to revolutionize healthcare, yet their safe adoption relies on effective Human-AI Interaction (HAAI) design and validation. Established usability engineering standards and guidances, including IEC 62366-1 and FDA frameworks, fail to address the novel sociotechnical risks of "black-box" systems, including automation bias and the misalignment of clinician mental models. This doctoral project directly addresses this gap, with the primary goal of developing a tailored human factor/usability engineering framework specifically for AIeMD. The project aims to establish robust methodologies for evaluating transparency and trust, integrating human factors and clinical performance into a multidimensional validation pipeline. Ultimately, this work will provide the evaluative tools necessary to move from subjective satisfaction to safety-critical, risk-based validation, ensuring that AI-enabled health solutions are clinically reliable, transparent, and compliant
Mariana de Oliveira· Information Hiding· 0 citations
Traditional usability assessments and questionnaires, such as the System Usability Scale (SUS), were designed for deterministic systems with predictable, linear outputs. However, AI-enabled medical devices are inherently probabilistic and co-evolve with the user through repeated interaction, rendering traditional usability assessments insufficient for guaranteeing the long-term safety in the use of high-risk probabilistic systems. Current literature reveals a striking absence of longitudinal studies, creating significant methodological blind spots regarding how trust calibrates over time and whether automation bias intensifies with habitual use. In this position paper, we present a manifesto for a longitudinal, three-fold methodological pivot in health human-AI interaction. We propose moving beyond static satisfaction metrics towards relational metrics —Longitudinal Trust Calibration (LTC), Automation Bias Drift (ABD), and Error Recovery Velocity (ERV)—that track the maturity and resilience of the human-AI partnership. This framework provides an actionable path toward a safety-in-use paradigm that acknowledges the temporal, dynamic nature of high-risk health AI.
Mariana de Oliveira, Célia F. Cruz, Nuno Matela· Information Hiding· 0 citations