Qi-Fan Deng

Gossamer: Distributed QoS-Aware Request Routing for Stateful Large Language Model (LLM) Inference at the Edge

This work proposes Gossamer, a distributed scheduler where each edge node routes requests using type-weighted scoring over gossiped partial state, and GLARE (Gossip-driven Local Adaptive Routing Engine), a metaheuristic that uses gossip staleness to control per-node weight adaptation.