Lightweight Model Inference Techniques in Edge Computing Environments: Review

ProfEdge: Efficient Construction of DNN Performance Evaluation Model on Edge Devices

With the rapid development of the Internet of Things (IoT) and artificial intelligence (AI) technologies, edge computing emerges as a crucial computing paradigm. By processing data near the source, edge computing enables faster and more efficient intelligent services. However, edge devices usually limit computational resources, and existing DNN inference latency profiling methods often rely on internal model details or large-scale latency measurements, making them costly and unsuitable for black-box deployment scenarios. This paper proposes ProfEdge, a fast construction framework for deep neural network (DNN) latency profiling models based on Gaussian process regression and Bayesian optimization. ProfEdge adaptively samples real latency measurements under different batch-size states and tunes profiling-model hyperparameters to reduce construction cost while improving profiling accuracy. Specifically, ProfEdge builds an adaptive sampling module based on Gaussian process regression to locate high-error regions through coarse-grained sampling and dynamically refine the sampling process. It further designs a dynamic Bayesian optimization mechanism to improve the accuracy of the latency profiling model. Finally, ProfEdge constructs a cross-device performance mapping model to migrate an existing profiling model to a target device with lightweight stratified calibration, thereby avoiding full reconstruction of the target-device profiling model. Experiments on various edge devices and DNN models, including CNN-based and transformer-based workloads, show that ProfEdge reduces profiling errors by up to 80% and saves over 70% of profiling construction cost compared with existing methods. The cross-device migration results further demonstrate that ProfEdge can achieve competitive profiling accuracy on new devices with only a small number of target-device calibration samples.

Scalable AI Inference Pipelines Across Edge and Cloud Computing Environments

A scalable edge-to-cloud AI inference pipeline in which inference tasks are dynamically distributed across heterogeneous edge and cloud resources is examined, providing a basis for resilient real-time AI systems while highlighting unresolved challenges involving heterogeneous hardware, dynamic workloads, privacy-utility trade-offs, and cross-layer optimization.

Decentralized Task Offloading for Multi-User Mobile Edge Computing with Machine Learning

A reinforcement learning-based computation offloading strategy using an improved Deep Deterministic Policy Gradient (IDDPG) algorithm that outperforms greedy offloading strategies and demonstrates improved efficiency in dynamic MEC environments.

Lightweight Deep Learning Models: Technologies, Applications, and Edge Deployment

Energy-Efficient Distributed Machine Learning in Edge Computing Architectures

This study investigates energy-efficient distributed machine learning techniques, including federated learning, model compression, adaptive resource management, dynamic task offloading, and communication-efficient optimization, and proposes a distributed learning framework that integrates local model training, adaptive communication scheduling, gradient compression, and workload balancing to minimize energy consumption while maintaining learning accuracy.

REVIEW OF APPROACHES FOR REAL-TIME PERFORMANCE GUARANTEES IN EDGE MACHINE LEARNING SYSTEMS

This article presents a structured and comprehensive review of approaches aimed at ensuring real-time performance guarantees in edge machine learning systems. It highlights the ongoing shift of artificial intelligence workloads from centralized cloud infrastructures toward edge environments, where data is generated and processed locally. While this transition improves latency, bandwidth usage, and data privacy, it also introduces critical challenges due to the limited computational resources of edge devices. The core contribution of the study is the development of a multi-layered taxonomy that organizes existing techniques into four key domains: model-level optimization, system-level scheduling, runtime adaptation, and hardware-software co-design. Each layer addresses different aspects of the real-time constraint problem. Model-level techniques focus on reducing computational complexity through methods such as quantization, pruning, and efficient architecture design. System-level strategies ensure timely execution through scheduling algorithms and resource allocation mechanisms. Runtime adaptation introduces dynamic adjustments to maintain performance under changing conditions, while hardware-software co-design aligns algorithms with the capabilities of underlying hardware. The article emphasizes that no single technique is sufficient to guarantee real-time performance. Instead, it underlines the importance of integrating multiple approaches to balance trade-offs between accuracy, latency, and energy efficiency. This holistic perspective is particularly relevant for safety-critical applications, where missing execution deadlines can lead to system failure. Overall, the study provides a clear conceptual framework and valuable insights for designing robust, real-time edge AI systems.