Evaluating the Accuracy of ChatGPT-4 in Editing Arabic Writing: A Comparative Study With Human Editors

Abstract

This study examines the accuracy of ChatGPT-4 in editing Arabic text as compared to human editors. Arabic language teachers and ChatGPT-4 edited a purposive sample of 25 texts written by freshmen, intermediate, and senior students at the Teaching Arabic Language Institute, Imam Mohammad Ibn Saud Islamic University. Grammatical, lexical, spelling, and stylistic errors were categorized using an error taxonomy, and accuracy was assessed using precision, recall, F1-scores, and error reduction rates. In order to contextualize perceptions of AI versus human editing, learner feedback was also collected. Findings indicate that there is a strong correlation between human ratings and ChatGPT-4 in grammatical accuracy (ρ = 0.81), resulting in an effective reduction of surface-level errors. However, it underperformed human ratings in stylistic flow and lexical refinement. Students described ChatGPT-4’s editing as clear and simplified but often mechanical, while human-edited texts were reported as more natural and culturally meaningful. Inferential analyses confirmed significant differences between the two conditions in stylistic and lexical dimensions. These results suggest that ChatGPT-4 has potential as a supplementary tool for Arabic language learning, particularly in areas such as grammar correction, fostering learner autonomy, and providing instant feedback. Nonetheless, the study highlights that for advanced writing abilities, stylistic richness, and cultural appropriateness, a hybrid AI–human model is necessary.