Evaluation and Validation of a Developed Hybrid Models for ERP System Usability and Its Influence on User Satisfaction in Enterprises

Abstract

Enterprise Resource Planning (ERP) systems have become indispensable for integrating organizational processes and improving operational efficiency. However, evaluating ERP system effectiveness remains challenging because conventional evaluation methods often rely on subjective assessment techniques and fail to capture the complex relationships among multiple organizational and operational performance indicators. This study proposes a hybrid framework that integrates Inverse Variance Weighting (IVW), Tabu Search Algorithm (TSA), and supervised Machine Learning (ML) models for objective ERP effectiveness evaluation. Operational and organizational Key Performance Indicators (KPIs), including throughput, lead time, defect rate, machine utilization, system uptime, response time, employee productivity, order fulfillment rate, supply chain responsiveness, customer satisfaction, cost efficiency, and return on investment, were employed to construct a composite ERP effectiveness index. The IVW technique objectively assigned KPI weights based on statistical stability, while Tabu Search optimized KPI selection and feature combinations. Linear Regression, Support Vector Regression, and Random Forest Regression models were subsequently trained to predict ERP effectiveness. Model performance was evaluated using Mean Absolute Error (MAE), Root Mean Square Error (RMSE), coefficient of determination (R²), and five-fold cross-validation. The proposed framework provides an objective, scalable, and intelligent decision-support mechanism capable of improving ERP performance evaluation in manufacturing environments. The study contributes a novel hybrid optimization and machine learning framework that enhances predictive accuracy while reducing the subjectivity associated with conventional ERP evaluation techniques.