Jun 2026
TAILOR: Token-Aware Partitioning and Routing for Edge–Cloud Transformer Inference
TAILOR is presented, a token-aware inference framework that jointly optimizes offline partitioning and online routing for autoregressive transformer serving that reduces end-to-end latency and lowers OOM-induced fallback compared with static partitioning baselines, demonstrating robust inference under long-tailed token workloads.
Xiaoyao Huang, Remington R. Liu, Jie Wu
· International Workshop on Qu... · 0 citations