Addressing SOTIF with a runtime monitor ensemble for AI-based perception systems
TL;DR
This work presents a conceptual and process-oriented framework that integrates an ensemble of runtime monitors into a SOTIF-oriented development process for AI-based environment perception systems and analyzes how these monitors may support selected SOTIF activities, including risk identification, verification and validation, and operational-phase feedback.
Abstract
AI-based environment perception systems are a key enabler of autonomous driving, yet assuring their safety remains a fundamental challenge. Just like human drivers, such perception systems will never be completely safe to use, in the same way as human driving will never be completely safe. Due to the inherent limitations of machine-learned models operating in open and only partially foreseeable environments, complete correctness cannot be guaranteed, making it necessary to argue the absence of unreasonable risk rather than absolute safety. To help with such a proof, the concept of Safety of the Intended Functionality was introduced with ISO norm 21448 (SOTIF), which concerns making a structured safety argument of the correct functioning of a system component of an autonomous vehicle in the operational domain it was designed for. The SOTIF norm defines a set of activities for this purpose, but leaves considerable freedom regarding how such a proof is made. In this paper, we present a conceptual and process-oriented framework that integrates an ensemble of runtime monitors into a SOTIF-oriented development process for AI-based environment perception systems.The proposed monitoring ensemble consists of complementary monitor types that address different sources of uncertainty in perception systems, including inconsistencies between redundant systems, deviations from known input–output behavior, and unfamiliar internal activation patterns. We analyze how these monitors may support selected SOTIF activities, including risk identification, verification and validation, and operational-phase feedback.Rather than providing an implementation or quantitative evaluation, this work focuses on the systematic integration of monitoring into the SOTIF lifecycle and its role in structuring safety arguments. The monitoring ensemble is intended to facilitate the identification of potential near-failure situations, support scenario discovery, and provide additional evidence that may be used when assessing residual risk. As such, it contributes to concretizing the activities in a SOTIF-based development process for AI-based perception systems.