Skip to content
Open access

Knowledge Distillation Method for Compressing Large Language Model of Power Risk Identification and Improving Deployment Efficiency

Aug 2026 · Advanced Electromagnetics · 0 citations

TL;DR

Experiments show that, after applying the proposed knowledge distillation method, the inference latency is reduced from 235 ms to a minimum of 26 ms, which is better than DistilBERT’s 35 ms, verifying the efficiency and practicality of the lightweight model in resource-constrained scenarios involving power-risk identification, electromagnetic sensing, and edge-based intelligent monitoring.

Abstract

Large language models demonstrate high precision in power-system risk identification; however, their massive parameter counts and high resource consumption hinder real-time deployment on resource-constrained edge devices used in electromagnetic sensing systems, wearable monitoring terminals, and compact power-monitoring devices. Achieving low-latency processing in distributed sensing structures and edge-based electromagnetic monitoring devices requires a significant reduction in computational overhead to ensure immediate detection of electrical hazards, abnormal equipment states, and potential power-system risks. This paper proposes a lightweight compression method based on hierarchical supervised knowledge distillation. Experiments show that, after applying the proposed knowledge distillation method, the inference latency is reduced from 235 ms to a minimum of 26 ms, which is better than DistilBERT’s 35 ms. The number of student model parameters is reduced to 4.3% of the teacher model, namely 14.5M versus 340M, while the classification accuracy reaches 89.4%, close to the teacher model’s 92.7%. The F1 score for the equipment failure category reaches 90.3%, verifying the efficiency and practicality of the lightweight model in resource-constrained scenarios involving power-risk identification, electromagnetic sensing, and edge-based intelligent monitoring.

Read PDF

Similar papers

Conference Jul 2026

A Machine Learning Approach to Estimating Energy Use in Language Model Inference

The rapid expansion of Large Language Model (LLM) serving in cloud data centers has created a critical need for energy-aware scheduling. However, estimating inference energy typically requires hardware-level power telemetry, which is rarely accessible to cloud tenants. This paper proposes a lightweight, machine-learnin...

Bediga Sharan, Swarup Ghosh · 0 citations
Open access Aug 2026

Knowledge Distillation and Quantization-Aware Compression for Remaining Useful Life Prediction on Resource-Constrained IoT Devices

Preliminary cross-domain experiments on a small plant-extraction dataset suggest that the same pipeline can regularize smaller data settings, indicating that runtime compatibility deserves validation as early as offline accuracy in IoT-bound RUL research.

Kurnianingsih, Nurseno Bayu Aji, Dwiana Hendrawati et al. · 0 citations
Open access Aug 2026

Application of BERT-Free Pre-Training Model in Identifying Power Safety Risk Points

A lightweight CNN-based pre-training model built on a BERT-Free architecture for efficient and accurate power safety risk identification and offering valuable support for electromagnetic infrastructure monitoring and reliable operation in modern power systems is proposed.

S.-W. Yu, Y.-M. He, G.-B. Ban et al. · 0 citations
Open access 2025

Foundation Model Distillation Techniques for Resource-Efficient Predictive Analytics

Experimental results demonstrate that FMDF significantly reduces model complexity while maintaining high predictive accuracy, scalability, robustness, and energy efficiency, making it a promising solution for resource-efficient predictive AI, edge intelligence, federated learning, digital twins, and next-generation int...

Ken Iverson · 0 citations
#edge computing Open access Sep 2026

Edge-Computing-Oriented Lightweight State of Charge Estimation Method for Energy Storage Batteries

A gate-aware pruning strategy is developed to evaluate hidden-unit importance by jointly considering gate weights and gradient sensitivity, enabling structural compression while preserving critical temporal information and substantially improving computational efficiency.

Wen-Qiang Huang, Ting He, Wen-Long Zhu · 0 citations
#small language model Open access Aug 2026

Construction of a Small Model Based on Large Model Knowledge Distillation in Anomaly Behaviour Recognition for Intelligent Connected Vehicles

Experimental results validate the feasibility of transferring knowledge from large models under low-computational-power constraints and provide a new technical pathway and engineering reference for recognising anomalies at the edge of intelligent connected vehicles.

Jian-Jun Zeng, Jian-Guo Wei, Ge Song · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.