Adaptive storage optimization for deep learning workloads in converged HPC-cloud environments: a reinforcement learning approach
TL;DR
A deep reinforcement learning framework is introduced that combines Proximal Policy Optimization and Soft Actor-Critic with a Transformer-based state encoder to predict workload phase transitions rather than merely respond to them and indicates that temporally aware storage control can improve the efficiency of distributed training pipelines in heterogeneous HPC-cloud environments.
Abstract
Storage systems remain a significant bottleneck in large-scale deep learning training. Most evaluation studies rely on synthetic benchmarks that do not capture the I/O heterogeneity present in real training pipelines. Automated configuration methods often treat parameter selection as stateless, overlooking the temporal structure of recurring training phases. These challenges are addressed using a converged HPC-cloud platform that integrates Lustre with S3-compatible object storage. Three workloads with distinct I/O characteristics were evaluated: ResNet-50 on ImageNet-1K, 3D U-Net on the Medical Segmentation Decathlon, and Whisper Large on LibriSpeech. Time-to-convergence was used as the primary metric. We introduced a deep reinforcement learning framework that combines Proximal Policy Optimization and Soft Actor-Critic with a Transformer-based state encoder to predict workload phase transitions rather than merely respond to them. The learned policies reduced checkpoint write latency by up to 74.6% and improved GPU utilization across all three workloads. The approach generalizes across heterogeneous I/O profiles without per-workload reconfiguration. The results indicate that temporally aware storage control can improve the efficiency of distributed training pipelines in heterogeneous HPC-cloud environments.