Direct GPU Energy Measurements of Batch-Level Sharing in Mixture-of-Experts and Recurrent Models
This work reports direct GPU energy measurements on two workloads that batch very differently, recurrent policies in GPU-vectorised multi-agent reinforcement learning (MARL) and mixture-of-experts (MoE) language models, and asks one question of both: can the rows of a batch be made to share computation, across the batc...