Deploying Large Language Model (LLM)-based AI agents at the network edge enables real-time task handling in 6G networks. However, resource heterogeneity and user mobility necessitate agent migration to maintain Quality of Service (QoS). Unlike stateless containers, AI agents encapsulate invocation histories, planning contexts, and memory stores, making their migration considerably more complex. To address this issue, we propose AMD, an adaptive AI Agent Migration framework that integrates conditional Diffusion models with reinforcement learning to jointly optimize latency and resource utilization under dynamic edge environments. Specifically, we introduce a diffusion model as a generative prior to produce high-quality global deployment plans, effectively avoiding the local-optima problem common in conventional reinforcement learning. A reinforcement learning-based module then performs online policy refinement to accommodate real-time environmental variations. We implement AMD on a distributed system built upon AgentScope and validate it across geographically distributed edge servers. Experimental results demonstrate that AMD reduces the average task latency by 3.9%–27.6% and improves resource utilization by up to 53.3% compared to baseline strategies.
Jie Gao, Xingdan Wang, Zhiqing Tang et al.· Tsinghua Science and Technol...· 0 citations
Advancements in edge computing and container technology have made it increasingly popular and convenient to deploy Large Language Models (LLMs) through containers at the edge. However, the limited GPU resources of edge servers make it impractical to retain the model in GPU memory for long periods due to the high memory cost, especially when they remain idle without user requests. Existing work unloads the entire idle models to reduce memory costs on edge servers, but reloading them introduces significant loading delays that affect task Quality of Service (QoS). Therefore, efficient management of idle models is a critical issue that has been largely neglected in existing research and requires urgent attention. To address this gap, this paper studies the problem of idle model management from the perspective of the trade-off between memory cost and loading delay under the QoS constraint. A novel layer-granularity model unloading method is proposed, which leverages the layered characteristics of the model. We formulate an online joint optimization problem to determine which layers to unload and when, and present a layer-granularity unloading strategy inspired by the ski rental problem to solve it. We implement a real system with layer-granularity unloading for LLMs on NVIDIA GPUs and validate the effectiveness of the proposed method. Experimental results show it effectively trades off memory cost and loading delay, improving overall performance by up to 39.6%.
Zhenzheng Li, Zhiqing Tang, Jianxiong Guo et al.· IEEE Transactions on Mobile...· 1 citation