Jul 2026· International journal of computer information systems and industrial management applications· 0 citations
TL;DR
The proposed hybrid pipeline includes structured pruning, INT8 quantization and task-specific knowledge distillation, which is benchmarked against standalone methods and reinforces the idea of upper bound projection based approach for accuracy-oriented, multi-level compression.
Abstract
With the growing deployment of state-of-the-art deep neural networks in safety-critical, embedded and edge-computing applications, there is a strong incentive to design models that achieve high accuracy while maintaining very limited computational and memory budgets. We present a systematic study in resource-efficient AI model design addressing two orthogonal strategies of structured parameter reduction and accuracy-aware compression. Based on experiments on 6 benchmark datasets Image Net, CIFAR-10, GLUE (SST-2 and MNLI), MS COCO and Squad 1.1 we evaluate and compare pruning, quantization-aware training (QAT), knowledge distillation (KD), low-rank factorization and neural architecture search (NAS) in a systematic manner. The proposed hybrid pipeline includes structured pruning, INT8 quantization and task-specific knowledge distillation, which is benchmarked against standalone methods. Empirical results show that the proposed hybrid gives 4.5–5.2× inference speedup, 6–8× parameter reduction but just −0.2 to −0.3 percentage points accuracy drop compared to full-precision baselines on vision and language tasks. Five contextual analytical tables, capturing performance across the parameters latency alone, energy consumption alone and cross-task accuracy reinforce that Pareto-optimal results are consistently achieved for this hybrid method. Comparison with fundamental earlier research including Han et al. [5], Hinton et al. [12], Jacob et al. [9], Hu et al. [16], and Sanh et al. Now, looking at [20], it reinforces the idea of upper bound projection based approach for accuracy-oriented, multi-level compression. These results have immediate application to large-scale AI deployment on resource-limited hardware platforms, allowing AI democratization with fidelity.
The rapid growth of deep neural networks has produced models that deliver state-of-the-art accuracy while imposing substantial computational, memory, and energy costs, which limits their deployment on resource-constrained platforms such as mobile devices, embedded systems, and edge accelerators. This paper investigates model compression and optimization strategies, namely magnitude-based and structured pruning, post-training and quantization-aware training, and response-based knowledge distillation, as mechanisms for improving the performance-efficiency trade-off of convolutional neural networks. A unified experimental framework is proposed in which a ResNet-based baseline model is compressed using each technique independently and in hybrid combination, and the resulting models are evaluated on accuracy, model size, floating-point operations (FLOPs), inference latency, and energy consumption. Experimental results, summarized across six comparative tables and four analytical figures, indicate that structured pruning at moderate sparsity levels (30-50%) preserves accuracy within 1-2 percentage points of the baseline while reducing model size by up to 2.6x, that 8-bit quantization achieves near-lossless compression with a 4x reduction in memory footprint, and that a hybrid pipeline combining pruning, quantization, and distillation achieves an 8.5x compression ratio with an accuracy degradation of under 5%. These findings confirm that combining complementary compression strategies yields substantially better performance-efficiency trade-offs than any single technique applied in isolation. The paper concludes with a discussion of open challenges, including hardware-aware compression, automated compression policy search, and compression of large-scale transformer architectures, and outlines directions for future research.
Upma Sharma Archana· International Journal of Res...· 0 citations
Deep neural networks (DNNs) have achieved outstanding performance in areas such as computer vision, speech recognition, natural language processing, and autonomous systems. However, their high computational cost, memory usage, and energy consumption limit deployment in resource-constrained environments like mobile and edge devices. Model compression has emerged as a crucial solution to improve efficiency while maintaining accuracy. This paper provides a comprehensive study of neural network compression techniques, including pruning, quantization, low-rank factorization, knowledge distillation, and neural architecture optimization. These methods are analyzed based on compression ratio, latency, memory efficiency, and accuracy trade-offs. The study also explores hybrid compression approaches and proposes a systematic workflow from model training to deployment on constrained hardware. Experimental results demonstrate that effective compression significantly reduces model size and computational cost with minimal performance loss. The paper highlights the importance of compression-aware design and concludes as a valuable reference for building efficient and scalable AI systems.
Daniel Rodríguez· International Journal of App...· 0 citations
Deep neural network compression focuses on reducing the number of parameters and computational complexity of neural networks, enabling their use in resource-constrained environments where smaller networks, faster response times, and lower energy consumption are critical. In this paper, we propose a novel compression method that uses Singular Value Decomposition (SVD) informed by data examples. Unlike existing SVD-based approaches that operate solely on network weights, our method leverages input data to more effectively preserve essential information during compression. This data-driven approach significantly enhances performance compared to state-of-the-art techniques. Through experiments on standard benchmark datasets, we show that our method achieves substantial reductions in model size and computation with minimal impact on accuracy. Furthermore, we demonstrate its applicability in compressing deep neural networks to a fraction of their original size, achieving competitive accuracy of networks with a compression ratio of less than 3× with no fine-tuning, and at most 25 epochs of fine-tuning for higher compression ratios.
Abdelfattah Toulaoui, H. Khalfi, I. Hafidi· Journal of Artificial Intell...· 0 citations
To address the large model size, high computational cost, and limited deployment resources of keyword spotting models on edge platforms, this study proposes a collaborative multi-compression framework for lightweight deployment. Built on LiteKWS-Net, an attention-enhanced 2-D convolutional backbone, the framework combines adaptive importance-aware structured pruning, mixed-precision quantization, and quantization-aware multi-stage knowledge distillation. The retrained teacher reaches 97.90% (mean, 100,813 parameters, 0.385 MiB). MPDQ reaches 95.53 ± 1.16% at 8.27× theoretical weight compression. AIASP reaches 97.59% at a 30% target and 43.9% realized sparsity. The final joint model reaches 96.82% and, under ideal packed sparse mixed-precision storage, has a 51.55× theoretical weight-compression factor relative to the FP32 teacher; sparse-index overhead is excluded. On a Jetson Nano, the TensorRT FP16 network-body benchmark reports 2.86 ms latency and 0.69 mJ per inference.
Junbang Jiang, Rui Pu, Jin Li et al.· Symmetry· 0 citations
The growing incorporation of deep learning into autonomous robotic systems has highlighted the importance of efficient inference on resource-constrained edge platforms. Moreover, License Plate Detection (LPD) is a key component of intelligent transportation systems, requiring high accuracy and real-time performance for practical deployment. However, deep learning detectors often remain computationally demanding for edge devices with limited resources. This paper proposes an edge-oriented optimization framework for YOLOv8-based LPD, combining structured pruning, INT8 quantization. Extensive experiments evaluate the trade-off between accuracy, computational complexity, and inference latency. Results show that combining pruning and quantization significantly reduce model size and computational cost while preserving detection performance. The optimized model achieves real-time inference exceeding 100 FPS on an Intel CPU, demonstrating its suitability for resource-constrained edge environments. These findings confirm that hybrid compression strategies enable efficient deployment of deep learning–based LPD systems in real-world applications.
S. Ameur, Amir Ismail, Xavier Clady et al.· International Conference on...· 0 citations
Deploying compact Transformer encoders on resource-constrained edge devices forces a conflict between reasoning quality and tight latency, thermal, and energy budgets, yet static compression ignores both the variance in input difficulty and the changing state of the hardware. We propose Neuro-Elastic, an adaptive inference framework that operates at two granularities: per-input sparsity (entropy-driven early exit and token pruning) and device-state-driven switching among pre-compiled mixed-precision model variants. Its central contribution is this hardware and software co-design, not the individual mechanisms, which are established. Three components realize it: Hardware-Aware Differentiable Mixed-Precision Quantization (HA-DMPQ), which learns layer-wise bit-widths by gradient descent within the target accelerator’s native instruction set; LoRAQuant, a split-precision low-rank strategy for on-device fine-tuning; and Dynamic Sparsity Induction (DSI), which cuts memory bandwidth through entropy-driven exits and cache-aware token pruning. We measure all latency and energy on an NVIDIA Jetson Orin Nano and use a Qualcomm Hexagon 680 simulator only for instruction-level validation, scoping our claims accordingly. On the MASSIVE intent-classification benchmark with an mBERT-base backbone, the Eco operating point cuts latency by 46.2% and energy by 54.5% at a 1.6 percentage-point accuracy cost (95.2% versus the 96.8% FP16 teacher), while the Balanced operating point stays within 0.3 points of the teacher (96.5%) and still cuts latency by 31.9%. All results are reported as mean ± standard deviation over five independent runs.
Wenbin Shang, Dai Teng, Tingjie Chen et al.· IEEE Access· 0 citations