Assessing English-Arabic translation of verb phrase ellipsis: A comparative study of Google Translate and ChatGPT-4o

Abstract

Verb phrase ellipsis (VPE) poses considerable challenges in translation from English into Arabic, especially when using automated systems where syntactic nuances are often unresolved. This study examines how Neural Machine Translation (NMT) tools and Large Language Models (LLM) process VPE in English-Arabic translation. Using Google Translate (GT) and ChatGPT-4o (GPT-4o) as representatives of each of these technologies, a dataset of 413 English sentences containing instances of VPE was translated into Arabic using both tools. Each output was then analyzed, focusing on accuracy and ellipsis recovery quality with a categorization of translation patterns used in rendering the VPE instances. This study aims to assess how GT and GPT-4o translate English VPE into Arabic, focusing on the accuracy of ellipsis reconstruction and the translation strategies employed. A quantitative comparative method was employed, supported by a frequency-based analysis to determine recurring patterns. The results revealed that both tools employed the same recovery patterns but with different frequencies. GT outperformed GPT-4o in overall accuracy, producing more consistent and contextually appropriate translations. GT successfully used modulation, lexical repetition, and substitution, while GPT-4o heavily used substitution but with many incomplete instances, indicating its lack of ability to make context-dependent inferences. The study underscores the importance of better-quality training data for NMT and LLM tools for both discourse-level processing and context-dependent data. This study offers insights into research on computational linguistics and machine translation with practical implications for translation software developers, Arabic language specialists, and researchers in machine translation assessment.