ModBench: A Pipeline for Building Modelica Benchmark Datasets Mined from Library Repositories
TL;DR
ModBench is a pipeline that mines Git repositories of Modelica libraries to produce benchmark datasets of model snapshots, spanning the full commit history (since Modelica language v3), with 85562 distinct class snapshots, and links enabling traceability to original models and Git metadata.
Abstract
Research on equation-based cyber-physical systems modeling languages, such as Modelica, is constrained by the lack of curated benchmark datasets. This limits empirical insight into the evolution and development of models. We address this gap with ModBench, a pipeline that mines Git repositories of Modelica libraries to produce benchmark datasets of model snapshots. The pipeline (1) filters repository commits to retain human-authored, Modelica-relevant revisions; (2) extracts simulation-eligible classes; and (3) builds canonical representations of Modelica classes. For empirical validation, we applied ModBench to the Modelica Standard Library (MSL) and report the resulting dataset, spanning the full commit history (since Modelica language v3), with 85562 distinct class snapshots, and links enabling traceability to original models and Git metadata. The dataset, its API, and the data generation pipeline are publicly available to support future research on model evolution analysis, compiler testing, and automated model repair or generation.