Measure What Survives: When Model Vocabulary Outlives Generated Code

Abstract

Evaluations of model-based engineering (MBE) typically report what a generator produces—coverage, artefacts emitted, lines generated. Yet under industrial delivery pressure much of that structure is later overwritten, bypassed, or absorbed into hand-written code, while other model-derived assets quietly endure. We argue the operationally meaningful question is not what a toolchain produces but what a delivered system retains—and that retention is readable directly from shipped code. A delivered model leaves a system two assets, structure and vocabulary, and each can fit the delivered code independently of the other: structural fit is conditional, while conceptual fit can persist after generated structure erodes. We introduce retention mining, a way to read a model’s payoff from the delivered system itself—scoring both by what was retained rather than what the generator emitted—operationalised by four repository-mined measures. On an industrial mobile health (mHealth) platform of heterogeneous backend services and a cross-platform client, the two axes come apart: more generated structure can accompany less retained vocabulary, yet model-derived names persist across every service and reach the client even where no structure did—so retention can reverse the verdict that production alone would report. We outline retention mining as an emerging method for studying the durable value of generative abstractions—model-based and, increasingly, LLM-based—across systems and over time.