TS-D3QS: A Traffic-State-Aware Dueling Double-DQN Scheduler for Adaptive Network Queue Control

Abstract

Adaptive queue management must balance throughput, delay, packet loss, and fairness under changing traffic and resource conditions. This paper proposes TS-D3QS, a traffic-state-aware queue scheduler that formulates multi-queue resource allocation as a discrete reinforcement-learning problem. The scheduler observes normalized queue and port features and selects one of 16 interpretable allocation profiles, each jointly specifying bandwidth shares, shared-buffer shares, and priority multipliers. A Dueling Double-DQN learner separates state value from action advantage and use a Double-DQN target to reduce value overestimation. Experiments in a reproducible four-queue simulator under bursty and non-stationary traffic show that TS-D3QS reduces latency by 7.54%, reduces aggregate loss by 3.21%, improves Jain fairness by 3.30%, and improves reward by 6.01% over vanilla DQN, while classical CoDel-like and WFQ-like rules remain competitive on selected objectives.