Advancing Machine Learning for Resource-Constrained Environments

Abstract

This dissertation addresses the fundamental challenge of enabling machine learning intelligence on resource-constrained systems. These systems include embedded edge devices, mobile platforms, and robots operating under both on-device and cloud-based paradigms. The resource-constrained systems must perform low-latency, memory-efficient, and privacy preserving intelligence close to users and their data. Such systems are essential for personalized on-device learning, autonomous task execution in everyday environments, and user-adaptive multimodal assistants. However, ML on resource-constrained devices face three characteristic challenges that distinguish it from traditional ML settings: (1) heterogeneous data that varies across devices, users, and time; (2) strict hardware budget with limited compute and memory; and (3) stringent runtime efficiency requirements under dynamic environments. Efficient ML system design must tailor to these characteristics. The dissertation first introduces a work which addresses heterogeneity across edge devices, where each device's local data distribution differs from others. It demonstrates that dynamically adapting model parameters per device and per data instance improves prediction accuracy across all clients. The second work addresses temporal heterogeneity, where each device’s local data distribution can change over time (concept drift). It introduces drift-aware optimization, which effectively maintains model performance as data evolves under a dynamic environment. Next, the dissertation tackles the challenge of strict memory constraints on resource-constrained devices using forward-mode automatic differentiation (FmAD) to train large language models. This method enables low memory consumption during training while achieving accuracy comparable to backpropagation-based methods. Building on insights from FmAD, the dissertation further examines the trade-offs of FmAD and other memory-efficient gradient estimation methods as alternatives to backpropagation. It uncovers constraints on the computation, runtime, and convergence of the gradient estimation methods that inform the design of future algorithms. Finally, it targets the challenge of runtime-efficiency under dynamic environments for multi-agent planning tasks, where several physical or simulated robots must coordinate to complete shared objectives. The dissertation introduces a runtime-efficient planning framework that combines symbolic constraints with LLM-based reasoning. This approach reduces plan failures and execution time, while improving task success in dynamic environments.