Chang-Sheng You

Latency-Aware Edge LLM Inference via Model Preloading and Semantic Knowledge Reuse

Deploying large language models (LLMs) at the network edge requires low-latency inference despite limited GPU memory and frequent model switching. Conventional caching strategies are ineffective because semantically similar queries may differ significantly in surface form. To address this issue, we propose a unified fr...