Evaluating ChatGPT as an AI-assisted tool for statistical analysis: A comparative study with SPSS
Abstract
As AI-assisted tools for statistical analysis and research processes, large language models (LLMs) are being investigated more and more. In comparison to IBM SPSS, this study assesses the cross-platform agreement and within-model repeatability of ChatGPT-assisted statistical analysis. Utilising ChatGPT, a previously published dataset with 426 observations was reanalysed utilising the statistical techniques used in the first SPSS study. The same dataset and analytical guidelines were employed in three separate ChatGPT sessions, and the outcomes were contrasted with the SPSS reference findings. Normality tests, chi-square tests, descriptive statistics, t-tests, Levene's tests, one-way ANOVA, and Welch's test were all part of the assessment. The results demonstrated significant consistency over the three ChatGPT sessions and high agreement for primary inferential statistics throughout the regular operations studied. Despite constant main test statistics and p-values, several derived denominator degrees of freedom fluctuated across sessions, and disparities were found in some normality and descriptive-statistics outputs. The analytical process was created and interpreted using ChatGPT, while statistical libraries based on Python were used to carry out numerical computations. Overall, under well-defined circumstances, ChatGPT-assisted analysis showed excellent repeatability. These results validate ChatGPT as an AI-assisted supplement to traditional statistical software, not as a substitute for it.
