Test Suite Generation for Simulink Models: An Experience with Reactis

Abstract

Testing Simulink models is a widely recognized, relevant, and challenging problem for automated software engineering. It is relevant to industry practitioners since Simulink is a development and simulation language widely used by the Cyber-Physical System (CPS) industry. It is challenging since CPS are usually complex, entail many components, and failures are not easily identifiable without automated support. This paper reports our experience in applying Reactis, an industrial tool for testing Simulink models, to the ten Cyber-Physical Challenges Simulink models from Lockheed Martin. We considered 48 requirements from these challenge models. We assess the effectiveness of Reactis in identifying failure-revealing test cases and its efficiency. We compare our results and those from an academic testing tool (HC), and two model checking tools (QVtrace and CoCoSim). Our results show that Reactis could return a failure-revealing test case for a requirement that was considered correct. They also show that the efficiency of Reactis is comparable to that of existing tools.