From Software Acquisition to Operational Capability: Measuring ERP-Enabled Performance: A Transaction-Level Benchmarking Study and Prospective Matched-Case Evaluation Framework for SMES And Government Vendors

Abstract

- Enterprise resource planning (ERP) projects are often evaluated at acquisition, configuration or go-live, although the economic and operational value of an integrated system depends on whether the organisation redesigns processes, governs data, embeds controls, develops user capability and measures realised benefits. This study develops an acquisition-to-capability framework and tests its operational measurement layer using a public 70,000-record analytical extract of the DataCo Smart Supply Chain dataset. The extract contains 44,026 unique orders dated from 2015 to 2018. Analysis was conducted at order and order-line levels using descriptive statistics, cross-segment heat maps, robust logistic regression, holdout prediction, calibration assessment, bootstrap confidence intervals and sensitivity testing of a six-component ERP Operational Capability Index (EOCI). The order-level late-delivery rate was 54.7%, severe delay affected 23.6%, negative-profit exposure affected 19.6%, and hard workflow exceptions affected 10.8%. Shipping configuration was the dominant observable delivery factor: compared with Standard Class, adjusted late-delivery odds were 34.22 times higher for First Class, 5.42 times higher for Second Class and 1.40 times higher for Same Day. A leakage-controlled holdout model achieved an area under the receiver operating characteristic curve of 0.731, a Brier score of 0.197, a calibration intercept of 0.008 and a calibration slope of 0.967. Observed on-time delivery rose from 11.2% in the lowest readiness quartile to 62.9% in the highest, a 51.6-percentage-point difference. EOCI rankings were stable under alternative weights (Spearman rho=0.998). The shipping-mode odds ratios reflect this dataset's own service-level design and are reported as a demonstration of what transaction-level analytics can reveal, not as a general finding about ERP-enabled delivery performance. The findings show that transaction visibility can diagnose capability gaps, but software records alone cannot establish implementation causality. The paper therefore specifies a matched difference-in-differences field protocol, KPI dictionary, governance gates and implementation playbook for SMEs and small government vendors. The central conclusion is that ERP value is realised through a governed operating system, not through software possession