Zoha Azimi Ourimi

DRLM: Deep Reinforcement Learning-Based LLM Query Orchestration in Edge Environments

Large language model (LLM) services increasingly process heterogeneous queries with diverse latency, accuracy, and resource requirements. While edge deployment reduces response time, the heterogeneity of devices and the diversity of model families, parameter scales, and quantization levels make efficient LLM query orch...