CEMA-Router: Infrastructure-Aware Batch Routing for LLM Inference in Cloud-Edge Systems
Large language models (LLM) are increasingly integrated into pervasive applications spanning heterogeneous cloud-edge resources. These heterogeneous resources differ in computational capacity, energy efficiency, network conditions, and deployed model quality, creating trade-offs among response quality, monetary cost, e...