Comparative Analysis of Digital Tools for Urdu-English Literary Text Translation: A BLEU Metric Evaluation

Abstract

This research conducted a comparative analysis of three digital translation tools, Google Translate, Lingvanex, and MateCat, evaluating their performance in translating Saadat Hasan Manto's Urdu short story "Khol Do" into English. The study utilized the BLEU (Bilingual Evaluation Understudy) metric to quantify the accuracy of each translation in relation to a human-translated reference translation by M. Umar Memon. Additionally, the research employed Molina and Albir's translation techniques to assess the quality of the translations. The goal was to identify the tool that consistently produced translations closest to the intended meaning and style of the source text or human translation. The findings indicated that Google Translate is the most successful tool in terms of both quantitative measures (having the highest BLEU scores) and qualitative aspects (having higher lexical fidelity and structural accuracy). The study highlighted the strengths and limitations of each digital translation tool in capturing the nuances of Urdu literature. The results suggested that while machine translation tools are improving, they still require human oversight to ensure accuracy and cultural relevance. The research contributed to the ongoing evaluation of machine translation tools and their applicability in translating literary texts, particularly from languages like Urdu, which posed unique challenges due to their linguistic and cultural specificities.