Improving low-resource neural machine translation by semantic distance augmentation

Abstract

Neural machine translation (NMT) has witnessed substantial advancements, leveraging its learning capability to deliver highly accurate translations. Nevertheless, the efficacy of NMT models is contingent upon the accessibility of extensive-scale, high-quality training data, and its performance suffers notably in the absence of such datasets. To tackle this challenge, we propose a semantic distance augmentation (SDA) method that integrates syntactic information from constituency parse trees into the NMT encoder to optimize self-attention. Specifically, the source language sentences in the training set are analyzed by constituency parse analysis and the semantic distance attention matrix is constructed. Then, a fusion strategy is designed to integrate this matrix into the self-attention weight, enhancing the representation of the source sentences. In addition, a SDA length-aware strategy is proposed to adaptively control the contribution of semantic distance in the attention computation. Empirical evaluations across multiple low-resource language pairs reveal that the SDA method achieves statistically significant improvements in translation quality over the strong baseline, without requiring additional training data or increasing model complexity.