Procyon, a fine-grain multi-tenancy framework that fuses the PE instruction streams of multiple workloads into a unified execution schedule, substantially reduces PE underutilization that results in 3× speedup over state-of-the-art sparse streaming accelerators and reaches a peak throughput of 61.2 GFLOP/s.
Abstract
Sparsity has become a defining characteristic of modern workloads, motivating the development of specialized accelerators to mitigate the challenges inherent to sparsity. Sparse streaming accelerators have proved to be an effective solution, yet current designs are restricted to single-workload execution. Additionally, they exhibit significant underutilization in processing elements (PEs) due to their non-zero scheduling strategies. To address these limitations, we propose Procyon, a fine-grain multi-tenancy framework that fuses the PE instruction streams of multiple workloads into a unified execution schedule. This allows instructions from different workloads to execute on the same PE, thereby, improving its utilization and enabling concurrent execution of multiple workloads on a single accelerator instance. We evaluate Procyon on AMD Alveo U55C FPGA using the SuiteSparse workloads and show that it substantially reduces PE underutilization that results in 3× speedup over state-of-the-art sparse streaming accelerators (Serpens and Chasoň), and reaches a peak throughput of 61.2 GFLOP/s.
To sustain the computational demands of modern ML and HPC workloads, programmable accelerators integrate large arrays of processing elements (PEs), typically built around single-issue in-order cores to maximize integration density and energy efficiency. As a result, each PE's instruction throughput is upper-bounded at...
Luca Colagrande, Pascal Etterli, L. Benini· 0 citations
Embedded computing demands high performance and energy efficiency across workloads with widely different computational characteristics. Choosing the most suitable execution model therefore requires a quantitative understanding of how each model maps to different workload classes. In this paper, we compare three executi...
Luca Colombo, Luigi Ghionda, Amirhossein Kiamarzi et al.· 0 citations
LMTracer is presented, a fine-grained and real-time performance profiling framework for production LLM services that embed profiling logic into the execution through graph-embedded probing and streaming buffered profiling data to CPUs on demand to keep the execution of user kernels uninterrupted.
Wei Liu, Yong-Chao He, Bo-Han Zhao et al.· Proceedings of the ACM SIGOP...· 0 citations
This paper evaluates four platform–mechanism combinations across a datacenter-class GPU and an edge computing platform, evaluating the capability of these mechanisms to guarantee bounded execution times and freedom from interference across both classical neural network architectures and more computationally demanding m...
Juan José Martín-Osuna, Carles Hernández, J. Flich· IEEE Access· 1 citation
Emerging artificial intelligence workloads, particularly Large Language Models (LLMs), place unprecedented demands on GPU memory systems. Consequently, architectural innovations increasingly target the memory system to alleviate these bottlenecks. However, existing works still rely on outdated memory hierarchy models t...
Zi-Wen Wang, Jia-Peng Zhou, Chang-Xi Liu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.