Automated Quality Assessment of Metamodel Evolution: A Case Study in Java Mutation Testing
TL;DR
Investigating the evolution of ModBEAM, a large-scale metamodel for Java bytecode, demonstrates how targeted metamodel evolution can improve both expressiveness and operational efficiency, and illustrates the benefits of automated quality assessment in guiding metamodel evolution.
Abstract
Metamodel evolution directly affects the foundations of model-driven approaches, yet its quality has rarely been assessed beyond subjective human judgment. In particular, automated methods for empirically evaluating the quality of evolving metamodels are lacking. This paper addresses this gap by investigating the evolution of ModBEAM, a large-scale metamodel for Java bytecode. The initial version of ModBEAM was designed to closely mirror the abstract syntax of Java bytecode. However, when applied to Java mutation testing, several limitations were exposed, including structural redundancy and the difficulty of expressing mutation operators as transformation rules. We describe the main evolution steps taken to overcome these limitations and systematically assess their impact. Our quality assessment includes the metamodel itself and related artifacts, such as instance models, transformation rules, as well as algorithms and tools based on the metamodel. The results demonstrate how targeted metamodel evolution can improve both expressiveness and operational efficiency, and illustrate the benefits of automated quality assessment in guiding metamodel evolution.