Hierarchical Reinforcement Learning with Hungarian Assignment for Reliable Urban Smart Metering Under Cognitive Spectrum Access

Abstract

Advanced metering infrastructure (AMI) is the sensing backbone of the smart grid, and its reliability underpins urban energy services such as state estimation, demand response, and distributed-energy integration. When AMI uses cellular spectrum leased through a cognitive mobile virtual network operator (C-MVNO), allocating channels to data aggregation points (DAPs) each frame is difficult because three uncertainties interact: imperfect spectrum sensing, time-varying and cross-channel-correlated primary-user activity, and stochastic urban propagation. Classical Hungarian assignment is optimal per frame but blind to primary-user dynamics, while cognitive-radio heuristics ignore queue state and cross-channel structure. We propose a two-timescale hierarchy that couples these established tools in a new way: a Proximal Policy Optimization (PPO) agent decides, once per epoch, which opportunistic channels to expose, and an exact Hungarian solver performs the per-frame DAP-to-channel assignment. To our knowledge this is the first coupling of a learned cognitive layer with exact Hungarian assignment for cognitive-radio resource allocation. On a 3GPP TR 38.901-compliant simulator, PPO significantly outperforms a Bayesian-belief baseline and the Hungarian-only configuration in delivery ratio, latency, and a strict per-meter satisfaction metric, and is robust across independent seeds and sensitivity sweeps. An architectural ablation shows the DAP tier is a precondition for viability, not merely an optimization.