Instruction-Tuned Translation Language Model Focused On Turkish

Abstract

Large language models have high computation and inference costs. Recently, Small Language Models (SLMs) have become more important because they require fewer resources and offer high efficiency. Different training approaches can be used for SLMs to achieve high performance, even on resource-constrained hardware.In this study, we present our MT-270M translation model. It was trained using instruction fine-tuning to provide high efficiency and success for bidirectional translation between Turkish and English. We observe how we selected the datasets for the training phase and how data selection affects translation quality. Then, we explain how we prepared our high-quality training data. Finally, we examine the effects of data quality changes and including different tasks in the training process on the success of the small language model.