Detection of Machine-Translated Croatian-English Phrases: Do Translators Agree in their Assessment?
TL;DR
The results suggest that even expert translators do not consistently reach the same conclusions when judging the origin of short, decontextualised translated phrases, and support the view that, at the phrase level, machine-translated output is increasingly difficult to distinguish from human translation on the basis of surface linguistic features alone.
Abstract
This study examines the extent to which machine-translated and human-translated Croatian–English phrases can be distinguished by expert evaluators by analysing inter-rater agreement. For this purpose, a parallel corpus was compiled from four Croatian research articles in the fields of Education and Psychology, together with their human translations and machine translations produced using the online tool onlinedoctranslator. com. Two professional translators, native speakers of Croatian with extensive translation experience, were asked to classify selected English translations of Croatian phrases as either machine-translated or human-translated. The analysis shows a raw inter-rater agreement of 75 % for both machine-translated and human-translated phrase sets, indicating a moderate level of agreement between the raters. These results suggest that even expert translators do not consistently reach the same conclusions when judging the origin of short, decontextualised translated phrases. The findings support the view that, at the phrase level, machine-translated output is increasingly difficult to distinguish from human translation on the basis of surface linguistic features alone.