Mixture-of-Experts (MoE) models are a compelling architecture for scaling model capacity, making them especially attractive for deployment on resource-constrained, single-GPU systems. However, this benefit is difficult to realize because expert parameters dominate memory, and token-level routing is dynamic, unpredictab...
Procyon, a fine-grain multi-tenancy framework that fuses the PE instruction streams of multiple workloads into a unified execution schedule, substantially reduces PE underutilization that results in 3 × speedup over state-of-the-art sparse streaming accelerators, and reaches a peak throughput of 61 .
Ubaid Bakhtiar, Jeremy Sha, Helya Hosseini et al.· 0 citations
This work categorizes LLM kernel operands into three inter-workgroup sharing patterns and shows that the required optimization strategies differ across categories, from simple per-workgroup pinning to subgroup-aware co-scheduling, highlighting the need for placement-aware kernel programming and smarter architectural su...
Donghyeon Joo, Sooraj Puthoor, N. Jayasena et al.· arXiv.org· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.