Jun 2026· NTU Journal of Engineering and Technology· Vol 5· 0 citations
TL;DR
A hardware-aware, system-level analysis of key optimization techniques, including pruning, quantization, knowledge distillation, Low-Rank Adaptation (LoRA), and Neural Architecture Search (NAS), shows that quantization consistently achieves the highest inference speedups and memory efficiency.
Abstract
Deep neural networks (DNNs), particularly large language models (LLMs), have achieved remarkable performance; however, they introduce significant computational and memory challenges that hinder practical deployment. This paper presents a hardware-aware, system-level analysis of key optimization techniques, including pruning, quantization, knowledge distillation, Low-Rank Adaptation (LoRA), and Neural Architecture Search (NAS). The study evaluates these methods in terms of scalability, hardware compatibility, and deployment efficiency. The results show that quantization consistently achieves the highest inference speedups and memory efficiency, while LoRA enables scalable fine-tuning with minimal resource overhead. In contrast, pruning provides limited real-world benefits unless supported by structured sparsity and specialized hardware. These findings highlight the importance of integrated optimization and hardware–algorithm co-design for efficient large-scale AI systems.
Deep neural networks (DNNs) have achieved outstanding performance in areas such as computer vision, speech recognition, natural language processing, and autonomous systems. However, their high computational cost, memory usage, and energy consumption limit deployment in resource-constrained environments like mobile and edge devices. Model compression has emerged as a crucial solution to improve efficiency while maintaining accuracy. This paper provides a comprehensive study of neural network compression techniques, including pruning, quantization, low-rank factorization, knowledge distillation, and neural architecture optimization. These methods are analyzed based on compression ratio, latency, memory efficiency, and accuracy trade-offs. The study also explores hybrid compression approaches and proposes a systematic workflow from model training to deployment on constrained hardware. Experimental results demonstrate that effective compression significantly reduces model size and computational cost with minimal performance loss. The paper highlights the importance of compression-aware design and concludes as a valuable reference for building efficient and scalable AI systems.
Daniel Rodríguez· International Journal of App...· 0 citations
The proposed hybrid pipeline includes structured pruning, INT8 quantization and task-specific knowledge distillation, which is benchmarked against standalone methods and reinforces the idea of upper bound projection based approach for accuracy-oriented, multi-level compression.
Krishna Kumar Tiwari, Komal Tahiliani, Uma Shankar Birthare et al.· International journal of com...· 0 citations
The rapid growth of deep neural networks has produced models that deliver state-of-the-art accuracy while imposing substantial computational, memory, and energy costs, which limits their deployment on resource-constrained platforms such as mobile devices, embedded systems, and edge accelerators. This paper investigates model compression and optimization strategies, namely magnitude-based and structured pruning, post-training and quantization-aware training, and response-based knowledge distillation, as mechanisms for improving the performance-efficiency trade-off of convolutional neural networks. A unified experimental framework is proposed in which a ResNet-based baseline model is compressed using each technique independently and in hybrid combination, and the resulting models are evaluated on accuracy, model size, floating-point operations (FLOPs), inference latency, and energy consumption. Experimental results, summarized across six comparative tables and four analytical figures, indicate that structured pruning at moderate sparsity levels (30-50%) preserves accuracy within 1-2 percentage points of the baseline while reducing model size by up to 2.6x, that 8-bit quantization achieves near-lossless compression with a 4x reduction in memory footprint, and that a hybrid pipeline combining pruning, quantization, and distillation achieves an 8.5x compression ratio with an accuracy degradation of under 5%. These findings confirm that combining complementary compression strategies yields substantially better performance-efficiency trade-offs than any single technique applied in isolation. The paper concludes with a discussion of open challenges, including hardware-aware compression, automated compression policy search, and compression of large-scale transformer architectures, and outlines directions for future research.
Upma Sharma Archana· International Journal of Res...· 0 citations
This paper thoroughly analyzes the evolution of parameter optimization strategies, ranging from initial methodologies to modern advancements, elucidating their principles and applications in natural language processing (NLP) and machine learning (ML), paying special emphasis to parameter-efficient fine-tuning approaches.
M. Najafi, Ehsan Tavan, Stefan W. Kuhn et al.· Artificial Intelligence Revi...· 0 citations
The design of transformer-based Large Language Models (LLMs) is being radically changed through new architectures that are able to overcome scalability limitations of previous designs, including Mixture-of-Experts (MoE), Multi-Head Latent Attention (MLA), and Multi-Token Prediction (MTP). As an open-weighted model released at the end of 2024, which has both state of the art architectural transparency and production scale efficiency, DeepSeeek-V3 represents the ultimate testing ground for investigating these modern technologies. This paper provides a comprehensive analysis of the architectural structure of DeepSeek-V3 based upon information from the DeepSeek-V3 Technical Report, industry benchmarking data and independent latency testing, to demonstrate how various techniques can be used to optimize training while still providing competitive performance in code generation and mathematical reasoning. In addition, latency testing conducted on a Distilled version of DeepSeek-V3, with approximately 14 billion parameters, running on a T4 GPU, reveals that although significant improvements have been made in optimizing latency there remains substantial barriers to deploying these models. Through this context, this research will serve as a reference document for practitioners and researchers who wish to understand current trends and challenges in increasing accessibility to high performance AI models.
Yassine Zouhdi, B. Hdioud· EPJ Web of Conferences· 0 citations
Large Language Models (LLMs) have brought transformative advancements to natural language processing. However, their massive computational and memory requirements—stemming from tens of billions of parameters—make deployment on resource-constrained edge devices such as mobile phones and embedded systems highly challenging. Consequently, extensive research efforts have focused on enabling efficient on-device LLM inference. This paper provides an in-depth survey of two major inference acceleration techniques for on-device LLMs: KV cache optimization and speculative decoding. We present the core principles, representative studies, and performance characteristics of each approach. Furthermore, we discuss open challenges and future research directions, aiming to contribute to the advancement of efficient on-device LLM systems.
Mingyu Jo, Sangoh Park· International Conference on...· 0 citations