ACDO-RL: Safe Reinforcement Learning for Joint Cloud Database Resource and Index Optimization

Abstract

Operating a cloud database requires continual decisions about compute capacity, memory and physical design under changing workloads, with the goals of meeting a latency service-level objective (SLO) at minimal cost. This paper presents ACDO-RL, a reinforcement learning (RL) framework that controls vCPU scaling, buffer-pool size and index creation in one discrete action space. ACDO-RL combines a Double DQN agent with workload-trend features and a safety shield that masks risky actions, and is trained with an explicit cost–latency–reconfiguration reward. Because no production system could be used for exploration, the framework is evaluated in a documented queueing-based simulator with daily cycles, bursts, workload-mix shifts and working-set growth. Over five training seeds and 30 unseen simulated days, ACDO-RL costs $22.1 per day, which is 38% below a tuned rule-based policy with an index advisor ($35.6) and 65% below peak provisioning, and it obtains the highest mean return among all methods (-197 versus -200 for the rule policy). The savings are bought with a higher SLO-violation rate (2.4% versus 0.6%), and under an unseen stress workload the advantage over the rule policy disappears. An ablation shows that the shield mainly reduces cost and cross-seed variability and that trend features mainly reduce violations. We discuss deployment conditions and limitations of simulation-based evidence.