This work introduces TANGCO (Topology-Aware Neural Graph-Guided Capacity Optimization), which uses a graph neural network policy trained through the cascade simulator with policy-gradient learning and a heuristic anchor to allocate a fixed capacity budget across nodes to resist cascades under local load redistribution.
Abstract
Networked systems, from power grids to traffic networks and cloud clusters, carry loads across nodes with limited capacity. A node whose load exceeds its capacity fails and sheds its load onto its neighbors, which can trigger a system-wide cascade. We study how to allocate a fixed capacity budget across nodes to resist these cascades under local load redistribution. The problem is difficult because no optimal allocation is known, and the fail-or-survive objective is non-differentiable and piecewise constant, so exact and gradient-based optimization methods do not directly apply. We introduce TANGCO (Topology-Aware Neural Graph-Guided Capacity Optimization), which uses a graph neural network policy trained through the cascade simulator with policy-gradient learning and a heuristic anchor. We evaluate TANGCO on five synthetic graph families and five real networks spanning power, road, air, and Internet topologies. The learned policy improves on the best of four hand-designed heuristics in all 450 synthetic instances and in 40 of 45 real-network conditions, with robustness gains ranging from 1.6% to 246%. The learned policies transfer to unseen graphs within a family and partially across related topologies, and TANGCO$^{pre}$, pre-trained on synthetic graphs, matches per-network training on unseen real networks. Training scales near-linearly with graph size, and TANGCO$^{pre}$ allocates on a new network with no per-target training, matching the deployment cost of a hand-designed heuristic. Free-vector variants without the GNN, stay close to the heuristics, so the graph representation carries the gain beyond numerical search. Finally, analysis of the learned allocations identifies when local risk is sufficient, leads to an improved closed-form heuristic, and reveals the regimes where a topology-aware learned policy remains necessary.
Networks with highly dynamic data transmission demands and network topologies are common in real world. A fundamental problem in such networks is achieving scalable traffic allocation to maximize long-term total throughput under link capacity constraints. However, state-of-the-art (SOTA) works lack scalability. This is primarily due to two reasons in large-scale networks: first, they require solving constrained optimization problems online, which leads to high decision latency; second, they rely on reinforcement learning algorithms for policy optimization, which are inefficient in exploration and challenging to train effectively. To address these issues, we propose the Fast Networked Control (FNC) policy framework, which firstly utilizes parallelizable neural network modules to process the state and generate raw decisions, followed by basic operations such as normalizations and comparisons, which do not require iteration or optimization, to obtain decisions that satisfy the constraints. Hence, FNC policy avoids solving constrained optimization problems and supports parallel execution, significantly reducing decision latency. Furthermore, this policy preserves gradient flow and supports backpropagation, which enable us to design an imitation learning algorithm to efficiently train the policy in an end-to-end manner. Experiments in large-scale networks show that our FNC policy achieves an average 8% improvement in demands satisfaction and 10 times reduction in decision latency versus SOTA works.
Zhaoxing Yang, Guiyun Fan, Anjie Cao et al.· IEEE Transactions on Network...· 0 citations
We address liquidity placement in the Bitcoin Lightning Network (LN): given a fixed budget, which channels should a node open to maximize its routing capacity? We cast this as a budget-constrained combinatorial optimization problem on graphs, selecting $k$ edge additions that maximize $s$--$t$ max-flow, a theory-grounded measure of routing capacity, and solve it with graph reinforcement learning. Our lightweight agent combines a message-passing policy network with proximal policy optimization (PPO) and action masking, and is trained under a hub-exclusion curriculum: the network's top hubs are removed from training subgraphs, forcing the policy to learn capacity-aware placement rather than hub attachment. In extensive experiments on real Lightning Network snapshots, our method consistently outperforms strong heuristic baselines on the max-flow objective across multiple seeds and unseen graphs. The agent has been deployed in production for peer recommendations, executing 4640 channel-open decisions that cumulatively allocate 267.3 BTC over $16 million across 30 managed nodes.
Harrison Rush, Vincent Davis, Simone Antonelli et al.· 0 citations
This work proposes Double-Channel Graph Attention (DCGA), an end-to-end reinforcement learning framework that isolates network reachability and demand-service logic into separate graph channels and constructs valid routes using a simulator-coupled, constraint-informed decoder.
GraphRoute-Transfer, a graph-neural-network policy that assigns per-node timers from local structural features and is by construction permutation- and size-invariant, is proposed, a graph-neural-network policy that assigns per-node timers from local structural features and is by construction permutation- and size-invariant.
Yuto Nakamura· Journal of Computing and Ele...· 0 citations
Traffic load balancing is a key near-real-time control function in dense and heterogeneous radio access networks, where uneven and time-varying traffic distributions can severely degrade resource utilisation and quality of service. Within the Open Radio Access Network (O-RAN) architecture, cell individual offset (CIO) control provides an effective mechanism to steer user handovers for load balancing. However, existing rule-based and centralised deep reinforcement learning (DRL) approaches suffer from limited adaptability and poor scalability, due to static assumptions, excessive signalling overhead, and the exponential growth of the joint action space. To address these challenges, this paper proposes a graph-based multi-agent reinforcement learning (MARL) framework with centralised training and distributed execution for CIO-based load balancing in O-RAN. Distributed actors perform decentralised near-real-time CIO control at individual RAN, while a centralised critic exploits graph-structured representations of inter-cell interactions during training. Graph attention mechanisms are employed to capture the heterogeneous and time-varying influence of neighbouring cells, improving learning stability and scalability. Simulation results demonstrate that the proposed approach achieves better performance than existing baselines.
Jinsheng Yuan, Mengbang Zou, Weisi Guo· International Mediterranean...· 0 citations
The integration of large-scale distributed energy resources (DERs) into distribution networks (DNs) brings challenges to the effective control of DNs. In traditional approaches, mathematical or reinforcement learning (RL)-based solution algorithms are commonly used. However, the exponential increase in the number of DERs reduces the effectiveness of these strategies. Mathematical methods struggle to cope with the dynamic uncertainty caused by the high penetration of renewable energy, while RL algorithms relying on global data training may violate multi-agent privacy protocols. This paper proposes a DNs cooperative optimization method based on resource aggregation and RL. To reduce optimization dimensionality and ensure the privacy of resource data, a dynamic aggregation strategy is employed to aggregate a large number of distributed energy resources into aggregated entities, and the adjustable active–reactive power boundaries of each aggregated entity are derived. To fully exploit the regulation capability of DNs, data centers (DCs), as novel devices, are considered as flexible loads. To improve the convergence speed of model training and decision-making accuracy, evolution strategies (ES) and prioritized experience replay (PER) are integrated into the Soft Actor-Critic (SAC) algorithm, respectively. The proposed method is validated on the IEEE 33-bus and IEEE 123-bus systems. The results demonstrate the effectiveness and superiority of the proposed method in ensuring the secure operation of DNs.