Quality Estimation for Machine Translation in Low-Resource Settings

Abstract

Evaluating the quality of Machine Translation (MT) for low-resource languages is a challenging cross-lingual problem in Natural Language Processing. Quality Estimation (QE) aims to predict MT quality in the absence of reference translations. This thesis investigates QE through multilingual modelling, architectural adaptations, novel data curation and efficient learning strategies across nine low-resource language pairs. First, this thesis analyses how multilingual training influences QE performance, showing that combining linguistically related languages yields consistent improvements over training on a single language pair. The research then examines the suitability of large language models (LLMs) for QE in zero-shot, in-context learning, and fine-tuned settings. While LLMs show broad generalisation ability, empirical analysis reveals limitations in low-resource languages, including inconsistencies in tokenisation and weaker cross-lingual alignment. Results indicate that prompt-based adaptation alone is insufficient for reliable QE for low-resource languages and motivate architectural refinement. Building on this finding, the thesis proposes ALOPE, a regression-aligned framework for adapting decoder-based LLMs by leveraging informative intermediate Transformer layers for QE. Combined with lightweight regression heads and parameter-efficient techniques, ALOPE enables smaller open-weight LLMs to achieve state-of-the-art performance on multiple language pairs while operating within memory and compute constraints. Further, this research extends QE beyond scalar prediction by introducing a dataset enriched with human-annotated error descriptions that provide contextual supervision. Building on this, we propose ALOPE-RL, a policy-based reinforcement learning framework that optimises quantised LLMs using multi-component rewards, achieving state-of-the-art performance and remaining effective under extreme data constraints. This thesis shows that QE for low-resource languages can be performed through linguistically grounded multilingual transfer, regression-aligned adaptation, and error-aware reinforcement learning, for practical translation evaluation. All datasets, models, and code are publicly released.