Meta-Twins: a system-of-systems digital twin for self-descriptive and autonomous data infrastructure

Abstract

Digital twins have become foundational in cyber-physical systems by mirroring asset state and enabling monitoring, simulation, and optimization. However, the data infrastructure that powers modern analytics and AI—including transformation logic, orchestration state, lineage, and data quality rules—is typically encoded across fragmented scripts, configuration artifacts, and logs, making it difficult to audit, evolve, or safely automate. This paper proposes Meta-Twins, a system-of-systems digital twin for data engineering infrastructure, where engineering intent is represented as normalized relational metadata (mirrored state) and continuously enforced by persistent daemon processes (autonomous behavior). Meta-Twins exposes a SQL-native control plane for defining transformations, dependencies, schedules, execution history, and quality gates, thereby decoupling semantics from tool-specific execution mechanics and providing a bounded interface for safe AI-assisted updates. We implement a prototype on Databricks to demonstrate feasibility and portability of the design, and we evaluate Meta-Twins through a structured comparison with procedural pipelines and declarative modeling tools across coupling, transparency, self-service readiness, and resilience. The results indicate that representing infrastructure intent as relational state enables consistent auditability, deterministic execution, and reduced operational coupling, supporting human-centric and resilient data engineering aligned with the conceptual goals of Industry 5.0, particularly in terms of transparency and human-centric interaction.