A Resource-Aware Asynchronous PBT Framework with Dynamic LoRA for Heterogeneous Edge Clusters
Abstract
Traditionally, training and hyperparameter optimization of deep neural networks rely heavily on centralized cloud data centers. However, privacy concerns are driving a paradigm shift to move model fine-tuning directly to localized edge environments. Migrating Population-Based Training (PBT) to the edge presents severe challenges: unlike uniform cloud servers, edge networks exhibit extreme hardware heterogeneity. While Low-Rank Adaptation (LoRA) accelerates individual trials on weak devices, permanently maintaining adapters causes structural divergence during PBT weight inheritance. To cope with these edge-specific constraints, we propose Dynamic LoRA-PBT, an asynchronous hardware-software co-design. Systematically, it mitigates extreme evolutionary staleness via a capability-aware scheduler and a late-stage CPU dropout mechanism. Algorithmically, it introduces a Merge-and-Unload strategy, injecting LoRA for early exploration and explicitly merging it into the dense model before mutation. Preliminary evaluations on an 11-node CPU/GPU edge-server cluster demonstrate a 23.4 percent reduction in Time-to-Accuracy (TTA). As an ongoing work, we are currently integrating Transformer architectures and conducting rigorous statistical validations to solidify this framework.