A cross-contextual meta-analysis of agentic AI in retail and supply chain operations: efficacy and implementation insights

Abstract

Abstract Agentic artificial intelligence (AI) systems are increasingly deployed in retail and supply chain operations, yet the empirical evidence for their efficacy has never been systematically quantified. This meta-analysis synthesizes 19 studies (total $$N = 5{,}292$$ ) to quantify the impact of agentic AI on operational performance and examine how contextual factors shape outcomes. Using a random-effects model with restricted maximum likelihood (REML) estimation and the Hartung-Knapp (HK) adjustment for confidence intervals, we found near-total heterogeneity across the included studies ( $$I^2 = 99.5\%$$ , $$\tau ^2 = 7.18$$ ), with a 95% prediction interval $$[-4.20,\ 7.36]$$ indicating that the true effect in any new setting could range from substantial harm to exceptional benefit. Against this backdrop of extreme variability, the pooled Hedges’ $$g = 1.58$$ (95% CI $$[0.26,\ 2.89]$$ , $$p = 0.022$$ ) should not be read as a general implementation expectation but rather as an average masking fundamentally divergent context-specific effects. Subgroup analysis by operational domain (retail, supply chain, e-commerce, inventory) identified domain-level differences under the common-effect model ( $$Q = 935.45$$ , $$df=3$$ , $$p < 0.001$$ ), though these differences were not statistically significant under the more appropriate random-effects model ( $$Q = 6.73$$ , $$df=3$$ , $$p = 0.081$$ ); domain estimate based on three or fewer studies (e-commerce, $$k = 3$$ ) are descriptive only and do not support inferential conclusions. The limit meta-analysis adjusted the estimate downward to $$g = 1.26$$ ( $$p = 0.066$$ ). These findings show that the current evidence, while statistically promising on average, is highly heterogeneous and subject to meaningful selective reporting. Practitioners should prioritize domain-specific pilots, distinguish simulation-based evidence from real-world evidence, attend to AI architectural differences, and interpret published effect sizes with considerable caution. Future research must prioritize real-world field experiments, architecture-specific evaluations, and publication of null results, along with the adoption of validated quality frameworks, to strengthen the reliability and credibility of the evidence base.