LMEdge: QoS-Aware LLM Inference Orchestration on Edge Clusters

CELLServe: An SLO-Aware and Cost Efficient LLMs Serving System for Serverless Computing Environments

CELLServe formalizes SLO-constrained joint resource provisioning as an optimization problem with a dedicated algorithm, and introduces an opportunistic instance merging strategy for decode phase functions to reclaim fragmented resources.

TAILOR: Token-Aware Partitioning and Routing for Edge–Cloud Transformer Inference

TAILOR is presented, a token-aware inference framework that jointly optimizes offline partitioning and online routing for autoregressive transformer serving that reduces end-to-end latency and lowers OOM-induced fallback compared with static partitioning baselines, demonstrating robust inference under long-tailed token workloads.

Workflow-Aware Expert Routing for Distributed LLM Serving Over the Edge-Cloud Continuum

Deploying Large Language Models (LLMs) over the edge-cloud continuum faces severe stability challenges due to the conflict between stochastic network topology and complex workflow dependencies. Existing schedulers, relying either on computationally prohibitive Graph Neural Networks (GNNs) or topology-agnostic heuristics, fail to reconcile this tension. To bridge these gaps, we propose STEM, a service-level and topology-aware orchestration framework that formulates distributed LLM serving as a workflow-aware routing problem over a monitored service overlay, in which heterogeneous service instances act as specialized experts. At the core of STEM lies the STAR-PPO algorithm, utilizing a lightweight graph-free perception mechanism. By leveraging Squeeze-and-Excitation attention, it extracts critical bottleneck features from raw telemetry with linear complexity, bypassing the scalability limits of message-passing paradigms. To further achieve Pareto-efficient trade-offs, we develop a Dynamic Weight Adaptation (DWA) mechanism that autonomously recalibrates optimization preferences based on entropy-regularized metric drift. Extensive experiments on real-world datasets spanning 2,000 nodes demonstrate that our framework significantly outperforms state-of-the-art baselines. Specifically, STAR-PPO reduces network transmission costs by 96.8% and improves comprehensive inference efficiency by 24.4%, while sustaining robust zero-shot generalization across regions, with average latency within $1.09\times $ of a target-domain-retrained reference under a strict cross-region protocol. Code and data are available at https://github.com/gymorsiback/STARPPO

Sparsity-Aware Scheduling for Efficient MoE-Based LLM Serving on Edge Servers

A Collaborative Edge Intelligence Framework for SFC Provisioning via Language Models

As Software-Defined Networking (SDN) and Network Function Virtualization (NFV) enabled networks scale in size and complexity, monitoring and managing Service Function Chains (SFCs) under stringent latency and resource constraints becomes increasingly challenging. Although Deep Reinforcement Learning (DRL) is widely applied to SFC provisioning and Virtual Network Function (VNF) placement, enhanced network state monitoring is crucial to capture unexpected network conditions and guide DRL agents toward more adaptive decisions. In this context, Language Models (LMs) enable flexible, natural-language (NL)–based, query-driven network monitoring; however, directly processing complex multi-metric NL queries is computationally expensive and error-prone. This paper proposes an end-to-end (E2E) edge-based query translation pipeline that decomposes multi-metric NL queries into simpler single-metric sub-queries. Query decomposition is performed using a retrieval-augmented language model (RAG-LLM) and compared with a lightweight rule-based decomposition baseline. The resulting sub-queries are translated into Structured Query Language (SQL) using FLAN-T5. A cloud-only baseline, which directly translates NL queries to SQL without decomposition, is also evaluated. The results show that the rule-based edge pipeline achieves the lowest latency, reducing E2E latency by up to 78% compared to RAG-LLM and 18% compared to cloud execution under high workloads. Under increasing arrival rates for the largest workload, the rule-based edge pipeline maintains superior performance over cloud, reducing total E2E latency by 57% at $\lambda = 0.8$ . While RAG-LLM provides greater flexibility for unseen query patterns, both edge-based approaches achieve 100% NL2SQL accuracy with zero decomposition failures, outperforming the cloud-only baseline (95% accuracy).

Performance modeling in FaaS workloads at the edge using machine learning

This paper proposes a Machine Learning-based framework designed to predict key performance indicators, including CPU utilization, memory, and energy consumption, based on incoming workload patterns, while simultaneously forecasting potential system overload conditions, and introduces a profiling methodology that characterizes serverless functions according to their resource consumption profiles.