Large language models have become increasingly capable agents for low-level code and kernel optimization, but isolated kernel benchmarks provide only a proxy for the deployment behavior that matters in language-model inference. We identify a benchmark-to-deployment gap: candidate kernels that appear correct and fast in standalone harnesses can exhibit different performance, safety, or phase behavior after integration into a real inference workload. We introduce LLM4LLM, a deployment-aware closed-loop optimization framework that starts from a target inference script, extracts phase-aware optimization tasks, searches with an experience-guided episodic agent, and accepts patches through in-model validation. Across ten language-model inference workloads on A100 and H100 GPUs, LLM4LLM improves end-to-end latency for every evaluated model, achieving 3.91$\times$/6.98$\times$ geometric-mean speedups on A100/H100; as supporting kernel-level evidence, it also attains up to 2.745$\times$ GeoMean speedup on KernelBench Level 2.
Hui Zeng, Pengfei Yang, Yanxin Chen et al.· 0 citations
Mobile edge computing (MEC) has emerged as a promising paradigm to support latency-sensitive applications by deploying computing and storage resources closer to end users. Service caching at the network edge is an effective approach to further reduce response latency and improve quality of service (QoS). However, the increasing scale and dynamic nature of user requests pose significant challenges to efficient service caching and updating under limited edge resources and cost budgets. In this article, we investigate the joint optimization problem of service caching and dynamic updating in a cloud–edge–end collaborative architecture, with the objective of minimizing the average service access latency of users. We first construct a comprehensive system model that captures communication latency, service response delay, and the costs associated with service caching and updates. The problem is formulated as a nonlinear integer programming problem and addressed via a two-stage solution framework. Specifically, in the initial caching stage, we employ Lagrangian relaxation to reduce the high-dimensional knapsack problem and design an efficient algorithm integrated with Tabu Search. In the dynamic updating stage, we develop a Monte Carlo tree search (MCTS)-based algorithm enhanced by service popularity awareness, along with a novel service replacement strategy that considers user preference and service reliability. Extensive experiments under various system scales demonstrate that the proposed approach significantly outperforms state-of-the-art methods in terms of average service latency, cost budget satisfaction, and update efficiency.
Tianyang Zheng, Pengfei Yang, Kang Shen et al.· IEEE Internet of Things Jour...· 0 citations