2026· Computers, Materials & Continua· 0 citations· 28 references
TL;DR
ComPIM is proposed, a processing-in-memory accelerator for compressed DNNs without decompression that achieves high compute-throughput as well as low memory usage under the given memory bandwidth.
Abstract
: The increasing size and complexity of modern deep neural networks (DNNs) introduce significant challenges for deployment on resource-constrained embedded systems. Though the quantization techniques reduce memory usage and computational complexity by replacing high-precision parameters with low-precision counterparts, they are not sufficient for meeting the strict resource constraints in edge environments. To further reduce the model size, the compression techniques have been proposed. However, these techniques typically incur significant performance overhead due to the need for decompression before computation. In this paper, we propose ComPIM, a processing-in-memory accelerator for compressed DNNs without decompression. ComPIM compresses the DNN parameters (i.e., weights and activations) by leveraging a DNN characteristic that the parameter values follow a normal distribution, and then performs DNN inference directly on compressed parameters. Thus, ComPIM achieves high compute-throughput as well as low memory usage under the given memory bandwidth. In addition, all the operations of ComPIM are lossless, which does not cause DNN accuracy degradation. Our evaluation results show that ComPIM improves the average compute-throughput and memory usage by up to 34.2% and 18.8% across various DNN models, respectively, compared to the quantized neural network based on INT8 format.
DCSR-GCN, a high-performance GCN accelerator based on dynamic compression and sparsity reordering based on dynamic compression and sparsity reordering, is presented and a two-phase reordering algorithm that combines conflict-aware row scheduling with reuse-aware column grouping is proposed, thereby mitigating RAW confl...
Jun-Sheng Chang, Yi-Min Zhao, Yu-Xin Huang et al.· ACM Transactions on Design A...· 0 citations
G-CasDec is presented, an optimized framework to enable efficient decompression on GPUs for general cascaded compression schemes that outperforms state-of-the-art designs with 2.8× higher compression ratios and 2.0× faster decompression, and is also extensible to arbitrary cascade schemes.
Yong-Qi Zhuo, Xin-Yu Zeng, Huan-Chen Zhang et al.· Proceedings of the ACM on Ma...· 0 citations
The rapid scaling of large language models (LLMs) has unlocked remarkable capabilities across various domains. However, deploying LLMs on mobile devices remains challenging because of their high computational and memory requirements. Low-bit quantization (e.g., 4-bit weights, 8-bit activations) can reduce model size wh...
Hang-Yu Zheng, Zhi-Hao Shu, Kun-Xiong Zhu et al.· Proceedings of the 32nd ACM...· 1 citation
Autoregressive decoding of large language models (LLMs) on edge devices is dominated by memory-bound general matrix--vector multiplication (GeMV) operations. Processing-using-DRAM (PUD) is a promising computing paradigm that can alleviate this bottleneck by reducing processor--memory data transfers and enabling massive...
Kevin Xu, Tatsuya Kubo, Daichi Tokuda et al.· 0 citations
The domain-specific hardware accelerators for deep neural network (DNN) inference have been widely adopted, particularly in resource-constrained edge computing scenarios but face a fundamental tradeoff among accuracy, on-chip resource utilization, and inference latency. Traditional compression techniques, such as pruni...
Chung-Mou Pan, Swarnalata Panigrahy, Salma Elmalaki et al.· IEEE Embedded Systems Letter...· 0 citations
Machine learning inference is moving to the edge due to latency, energy, and privacy constraints. The growing complexity of edge workloads places increasing pressure on memory hierarchies, making data movement a major limitation for both performance and energy efficiency. Fixed-function accelerators inherit the memory...
Vincenzo Petrolo, Flavia Guella, G. Masera et al.· Computers· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.