From Model Explainability to Operational Transparency in Agentic AI: A Transparency-by-Design Framework for Critical Energy Systems Under the EU AI Act

Abstract

Agentic AI can coordinate distributed energy resources, invoke tools, revise plans, and act through multiple interacting agents. In critical energy systems, however, model-level explainable artificial intelligence (XAI) cannot reveal how goals, constraints, tools, delegations, authority, human intervention, and system changes combine to produce an operational action. This creates a regulatory and engineering gap wherever EU AI Act requirements for transparency, record-keeping, and human oversight apply. The transparency-by-design framework addresses this gap by treating operational transparency as a compositional property created through evidence continuity across model, agent, interaction, system, and lifecycle levels. It translates five regulatory transparency functions into eight components, a seven-stage gated lifecycle, stakeholder responsibilities, evidence artefacts, and acceptance criteria. Together, these elements specify what must be transparent, to whom, when, and how adequacy should be assessed and maintained. Regulatory analysis and thematic synthesis of 101 studies provide the evidence base. Application to an agentic virtual power plant demonstrates how forecasts, goals, plan revisions, inter-agent decisions, operator interventions, execution records, and system versions can be joined within one reconstructable evidence chain. The framework extends XAI from local model explanation to lifecycle-wide operational transparency for compliance-oriented development, without claiming legal conformity or field effectiveness.