This work proposes RelayMoE, a ring-based MoE execution model that computes locally as expert weights or tokens circulate, avoiding full top-$k$-expanded dispatch buffers and achieves a $2\times$ average speedup over Megatron-LM.
Arnab Kanti Tarafder, Jaume Guasch-Martí, Gokcen Kestor et al.· 0 citations
GroupMask is proposed, which generates the group selectors of all layers with a lightweight hypernetwork, relaxes them with a Gumbel-Sigmoid parameterization and a straight-through estimator, and learns them through sparsity-budget regularization and self-distillation while keeping the pretrained weights frozen.
Zhen-Gao Li, Shuo-Qiu Li, Xiao-Fan Zhang et al.· 0 citations
TuRTLe, a unified evaluation framework designed to systematically assess LLMs across key RTL generation tasks, is proposed, enabling a comprehensive assessment of LLM performance in syntax correctness, functional correctness, synthesis, PPA optimization, and exact line completion.
Miquel Albertí-Binimelis, Cristian Gutierrez-Gomez, D. García-Gasulla et al.· ACM Transactions on Design A...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.