Considering the Intended Behaviour in Evaluating the Rule Compliance of MASS

Abstract

There is a rising interest in the development and use of Maritime Autonomous Surface Ships (MASS). To reduce risks in MASS operation and interaction amongst each other and with human operated vessels, safety needs to be evaluated before deployment. Vessels of all degrees of automation need to be able to comply with maritime traffic rules, such as COLREG rules, to avoid accidents and be predictable for the human operators of the vessels the MASS interacts with. A method to evaluate a MASS for rule compliance is the approach of scenario-based testing. Here, the MASS performs a set of scenarios, which are deemed relevant to demonstrate rule compliance, and each performance is then evaluated against the corresponding requirement derived from the rule. A scenario is passed, if the MASS acted in compliance with the rule and failed if the behaviour was in violation of the rule. To objectively evaluate the rule compliance during scenario-based testing, there is the need to objectively specify the requirements stated by the rules. In previous work we specified COLREG rules regarding behaviour in and around TSS (Traffic Separation Schemes) using MTSCs (Maritime Traffic Sequence Charts). We implemented specifications of TSS rules in our monitoring tool, that acts as a plugin for a maritime traffic simulation, with the intention to support the evaluation of rule compliance of simulated MASS’s behaviour. A limitation of that monitoring tool is, that while it assesses whether a MASS’s behaviour is satisfying or violating the specification it is evaluated for, the results imply nothing further about how satisfying this particular specification can be interpreted. There might be instances where a MASS is satisfying the specifications for more than one rule, or instances where a MASS is satisfying the specifications of one rule but not the specifications of the rule it intends to follow. In this work we discuss means of interpretation for the evaluation of simulated MASS’s behaviour in regard to MTSC specifications. In particular we describe how to include knowledge about the MASS’s intended behaviour (we here refer to it as the MASS’s intention) and how to interpret the results of our monitoring tool in terms of the intention. We demonstrate how considering the MASS’s intention is helpful for deriving a meaningful interpretation for the results of an evaluation of rule compliance. Including the knowledge about intention allows us to not only evaluate if any specification has been satisfied but to determine if the MASS’s intended behaviour has been performed correctly. We show with a set of examples regarding COLREGs for TSS, how including intention enables our monitoring tool to provide an assessment about whether the rule concerning the intended behaviour was followed correctly by the MASS.