AI-Based Optimization Models for High-Performance Computing and Intelligent Network Architectures

Abstract

Artificial Intelligence (AI) has become a necessary technology to optimize High-Performance computers (HPC) systems and intelligent network architecture, such as optimizing the use of resources, the schedulization of workloads and network efficiency. This paper discusses how the three AI-based optimization tools, i.e., Deep Reinforcement Learning (DRL), Long Short-Term Memory (LSTM) and Extreme Gradient Boosting (XGBoost) can be optimized to enhance performance in the HPC environment. It was done using a quantitative, simulation based, approach on the benchmark HPC workload and HPC network datasets. Data cleaning, feature selection, normalization and train-test splitting were used to preprocess the datasets and then model implementation and evaluation took place. The models were evaluated based on performance such as accuracy, precision, recall, F1-score, execution time, resource consumption, throughput, latency as well as the energy consumption. Experimental results demonstrated that the DRL model achieved the highest prediction accuracy of 96.8, 93.8 CPU resources consumption, 96.1 network efficiency, 12.8 Gbps throughput and 5.3 ms (lowest) network latency values. LSTM had the highest prediction accuracy of 95.2, and XGBoost had a score of 93.9 with the lowest training time rate of 39 minutes. The findings indicate that AI-based optimization is tremendously more effective regarding calculations and adaptability, scheduling, and resource allocation compared to the conventional ones. The suggested framework offers a viable solution to scalable, smart and energy efficient, intelligent HPC systems, which can be used to support next generations of computing and networking systems.