Aug 2026· Cluster Computing· Vol 29· 0 citations· 76 references
Computer Science
TL;DR
This work presents PROFET, a system that predicts the training latency of arbitrary Convolutional Neural Network implementations across a wide range of GPU types and mini-batch sizes and introduces a novel operation-name clustering heuristic that effectively resolves naming inconsistencies in computational graphs where semantically similar operations are labeled differently.
Deep neural networks (DNNs) are widely adopted in various fields, driving an emerging trend in developing software stacks associated with DNN training systems. For example, many codes have been ported across different frameworks or developed to leverage the computing power of GPUs or domain-specific accelerators. However, validating a kernel implementation in DNN training is time-consuming and generally requires massive storage. Specifically, this poses a fundamental question: how to characterize the behavior of a new implementation when it is integrated into a DNN training flow. Unfortunately, this problem is not well investigated in the literature, to the best of our knowledge. To address this shortcoming, we present PEAT - a lightweight inspection framework for \underline{P}seudo-\underline{E}rror \underline{A}ssessment associated with GPU kernel validation in DNN \underline{T}raining. Firstly, inspired by conventional fault injection (FI), PEAT's Profiler invokes an operation-wise kernel in a training flow to collect a DNN model's states (e.g., checkpoints and activations). More importantly, the Profiler introduces two simple yet effective techniques, playback FI and frequency-based runtime FI, leveraging persistent kernel calling during the training process. Secondly, PEAT's Analyzer characterizes profiled errors, revealing some signatures from the error distribution of a kernel compared to the golden one. Lastly, PEAT's Detector provides some guidelines as a sufficient condition, which enables associating several well-known error models with signature patterns. We demonstrate the applicability of our approach by presenting the results and analysis using GPUs from the two most popular vendors, NVIDIA V100 and AMD MI250, on various AI models, from vision tasks to language models, for both pretraining and finetuning scenarios.
Xuan Truong Nguyen Department of Next Generation Semico Convergence, Open Sharing System, Seoul National University et al.· 0 citations
A deep learning-based failure prediction model that integrates Convolutional Neural Networks and Bidirectional Long Short-Term Memory networks to identify job failures before they occur is presented, improving the performance of cloud computing applications by reducing job failures and optimising resource utilisation.
Wunukhen Shehu Awudu, P. Asuquo, B. Agbor et al.· E3S Web of Conferences· 0 citations
A dynamic recurrent neural network is proposed to accurately predict workloads and integrates an auto-encoder to effectively extract representations from the original workload data with high dimensionality to enable adaptive and accurate predictions for highly variable workloads.
Okore Kalu, C. Okafor, P. Asuquo et al.· E3S Web of Conferences· 0 citations
With the explosive development of LLM-empowered agent technology, LLM inference performance has become more important than training. Cloud computing is a popular deployment approach, where performance prediction is vital for instance selection and QoS assurance. However, prediction is challenging due to GPU hardware heterogeneity, Transformer operator variations, and dynamic inference configurations. Virtualization and other features vary across clouds, further increasing prediction difficulty. Existing methods suffer from low accuracy and poor generalization. To tackle these issues, we propose Dispeller, a prediction model for GPU-accelerated cloud environments with three feature sets: 1) basic GPU hardware feature with 7 dimensions; 2) operator-level GPU performance feature with 4 dimensions; 3) inference configuration feature with 4 dimensions. We conduct experiments on public cloud GPUs and collect a real-world dataset of 10,112 samples. Random Forest is adopted to learn the nonlinear mapping between features and performance. Experimental results show that Dispeller achieves high prediction accuracy with TPS $\mathrm{R}^{{2}} = 0.951$ on seen GPUs and 0.989 on unseen GPUs, demonstrating strong cross-GPU generalization. An ablation study confirms that inference configuration features contribute 87.3% of the predictive power. Dispeller is therefore able to recommend cloud resources and optimize LLM deployment costs.
Huan Zhou, Zhi-Peng Wang, Meng-Juan Li et al.· Fall Joint Computer Conferen...· 0 citations
Accurate pre-deployment estimation of CNN inference cost--energy, latency, and peak memory--is increasingly critical as models are deployed on resource-constrained GPU platforms. Existing approaches rely on FLOPs, latency measurements, or single-device profiling as energy proxies, overlooking the non-linear interactions between architectural design and hardware load. We present a workload characterization study of 13 419 CNN configurations on two GPU platforms (RTX 5090 and RTX 3080) under GPU telemetry, revealing that energy, latency, and memory exhibit fundamentally distinct scaling behaviors: energy and latency diverge by 3x under high computational demand, and cross-GPU transferability differs by target--energy and latency require platform-specific models while memory transfers well across the two tested platforms. Building on these characterization findings, we develop CARB, a cascade-blended ensemble that jointly predicts all three targets with R2 ~0.99, and a two-stage deployment screening workflow that eliminates over 90% of candidates in seconds, reducing large design spaces to a Pareto-prioritized shortlist validated against real hardware.
Conventional compute-in-memory (CIM) deployment flows usually assume a fixed crossbar geometry, although DNN layers often have different channel counts and matrix shapes. The resulting shape mismatch leaves part of the array capacity unused and can increase the number of split-and-transfer operations during inference. This paper presents FlexiCIM, a compiler framework for neural network deployment on multi-core CIM systems. FlexiCIM groups physical cores into virtual computing units (VCUs), maps reshaped layer weights under CIM capacity constraints, and schedules dependent tasks while accounting for communication overhead. The framework also uses an evolutionary search procedure to select VCU partitions for latency-first or throughput-first objectives. Experiments on five CNN models show that FlexiCIM achieves an average utilization of 75.5% under the default setting. In the latency-first setting, FlexiCIM reduces latency by up to 54.8% and obtains the lowest normalized energy among the evaluated designs (1.00 vs. 2.02/1.37/1.60 for Fixed-L/M/S). In the throughput-first setting, FlexiCIM provides the highest throughput on all five models, with $1.07\times-1.38\times$ speedup over the best fixed-size baseline. These results indicate that compiler-managed VCU partitioning and capacity-aware mapping improve deployment efficiency on multi-core CIM architectures.
Kaiwen Deng, Sifan Sun, Hanjie Liu et al.· IEEE Non-Volatile Memory Sys...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
