Continuous assurance for AI-driven clinical decision support systems

Abstract

Healthcare systems are rapidly embedding adaptive and generative AI into core clinical processes. The integration of Artificial Intelligence into Clinical Decision Support Systems (AI-CDSS) highlights a fundamental transformation within healthcare delivery. This transformation enables advanced predictive analytics, multimodal data integration, and real-time augmentation of clinical decisions. However, AI introduces systemic, ethical, operational, and governance risks that challenge traditional healthcare audit and assurance frameworks. In fact, assurance methodologies designed for static software are becoming insufficient for patient safety and regulatory compliance. This review discusses the evolving landscape of AI-CDSS audit, highlighting its transition from a technically focused lifecycle validation to an integrated paradigm centered on socio-technical resilience. Early audit approaches adapted conventional medical device and software validation models to machine learning–enabled clinical tools. While these models established baseline safety oversight, they were not designed to address adaptive algorithms functioning in complex, evolving clinical environments. Modern governance frameworks emphasize continuous performance monitoring, lifecycle surveillance, and structured human oversight. However, a persistent implementation gap remains. Many audit models poorly capture the interactions among algorithmic behavior, clinical workflows, organizational culture, and shifting patient populations. In fact, three major paradigm shifts are reforming the AI-CDSS audit. First, the audit scope is expanding from model-centric evaluation to ecosystem-level assurance that incorporates workflow integration, human-machine collaboration quality, and organizational learning capacity. Second, the field is moving from post-hoc explainability toward reasoning traceability and synergistic clinical sense-making. Third, audit philosophy is shifting from static compliance verification toward resilience-oriented monitoring that focuses on adaptive capacity, graceful degradation, and safe performance evolution. Thus, we propose the STRAICS framework (Socio-Technical Resilience Assurance for Intelligent Clinical Systems), which integrates technical robustness, human-machine interaction safeguards, adaptive governance, and transparency-by-design infrastructure. These components are vital for building trustworthy, effective, and impartial clinical AI ecosystems that can safely manage the increasing complexity of healthcare.