Development of a multimodal english cross-language translation model based on transformer architecture

Abstract

Translation across multiple forms and different languages shows limitations from problems with combining features and problems with moving meaning between languages, and these problems affect how the approach performs in settings that involve various conditions. This study aims to design a unified multimodal Transformer architecture that strengthens cross-linguistic semantic alignment and improves translation stability under heterogeneous input conditions. This study conducted initial processing that combined image and text data, allowing separate encoding of features at higher levels. It then used attention across different forms to combine meaning from multiple sources, producing a representation that showed unity. The approach also introduced space for meaning that multiple languages share at a level beyond surface forms, thereby improving the stability of the mapping between languages in the results. Training without modality and applying noise-enhancement techniques together improved this model’s robustness to incomplete and degraded input conditions that approximate practical deployment scenarios.The model was trained and evaluated on a structured English-German bilingual vision-language dataset composed of aligned image-text pairs collected under controlled experimental conditions.Experimental evaluation in fusion mode yielded BLEU scores of 39.2, METEOR scores of 31.7, and CIDEr scores of 113.8, indicating stronger semantic consistency and improved bilingual generation quality. The results of this research demonstrate that multimodal fusion and cross-language alignment mechanisms offer effective approaches to improve the accuracy and reliability of multimodal translation systems. The findings remain constrained by dataset scale and language coverage, and future research will extend validation across broader linguistic domains and more diverse multimodal scenarios.