Skip to content

COILD: An Indic-Centric Parallel Corpus and Benchmark for Machine Translation Across Indian Languages

Sep 2026 · arXiv.org · Vol abs/2609.28826 · 0 citations · 16 references
Computer Science

TL;DR

COILD, an Indic-centric parallel corpus comprising over 1.16 million human-translated and human-verified sentence pairs, covers 20 Indian language pairs across the Indo-Aryan, Dravidian, Tibeto-Burman, and Austro-Asiatic language families, and provides a valuable training and evaluation resource for advancing multilingual machine translation and future multilingual language models for Indian languages.

Abstract

Machine translation (MT) for Indian languages remains constrained by the limited availability of high-quality, Indic-centric parallel corpora and evaluation benchmarks. Existing multilingual resources are largely constructed from English-pivot content and often fail to capture the linguistic diversity, cultural complexity, and domain-specific characteristics of Indian languages. We present COILD, an Indic-centric parallel corpus comprising over 1.16 million human-translated and human-verified sentence pairs, covering 20 Indian language pairs across the Indo-Aryan, Dravidian, Tibeto-Burman, and Austro-Asiatic language families. The corpus is built entirely from original Indian language sources collected from licensed repositories spanning eight domains with direct real-world applicability. Furthermore, we introduce a domain-centric benchmark comprising 2,000 expert-verified sentences to enable consistent multilingual and cross-lingual evaluation across Indian language pairs. To validate the effectiveness of COILD, we fine-tune two representative multilingual neural machine translation models, IndicTrans2-Distilled and NLLB-200. Experimental results demonstrate consistent improvements across language pairs, domains, automatic evaluation metrics, and human evaluation, highlighting the effectiveness of high-quality Indic-centric supervision. COILD provides a valuable training and evaluation resource for advancing multilingual machine translation and future multilingual language models for Indian languages.

View source

Similar papers

Open access Sep 2026

Zero-Shot Evaluation of Multilingual Models for Bidirectional English–Indic Machine Translation

A detailed comparative analysis of five state-of-the-art models in a zero-shot translation paradigm between English and four typologically different languages, Indic languages: Gujarati, Malayalam, Hindi, and Tamil indicates that there is significant performance difference among language pairs, translation directions a...

Vishal Chhabra, Atul AgrawaL · 0 citations
Open access Sep 2026

Bridging the linguistic divide: recent developments in machine translation for Indian languages

This paper analyses various recent state-of-the-art variants of large language models (LLMs) and neural machine translation (NMT) for Indian languages in comparison to statistical machine translation (SMT) and tackles key questions, such as idiomatic expressions, morphologically complex grammar or the scarceness of par...

Jayanand A. Kamble, Shivajirao M. Jadhav, V. J. Kadam · 0 citations
Open access 2026

AssamLegalTrans: A Parallel Corpus, Benchmark and Analysis for English-Assamese Machine Translation of Legal Judgments

This work introduces the first-of-its-kind English-Assamese parallel corpus for the translation of Indian court judgments and performs a comprehensive evaluation of state-of-the-art multilingual models in both zero-shot and fine-tuned settings, comparing their performance against commercial systems.

Telem Joyson Singh, Hemanta Baruah, Sanasam Ranbir Singh et al. · 1 citation
#natural language process... Preprint Aug 2026

EnSiTa - A Trilingual Multi-Domain Parallel Dataset and Benchmark for Domain-Specific Machine Translation

EnSiTa is presented, a trilingual multi-domain parallel dataset and benchmark for English, Sinhala and Tamil, and is the most extensive systematically documented multi-domain parallel data creation and benchmarking effort for low-resource MT.

Surangika Ranathunga, Nisansa de Silva, Aloka Fernando et al. · 1 citation · ⚡1
Open access Oct 2026

Resource Sufficient Data Generation Technique for Machine Translation Models Using Named Entity Transliteration and Cross- Lingual Word Embeddings

Machine translation offers speed and cost advantages, but human translation remains superior in quality and contextual understanding. For low-resource languages like Hausa, the severe scarcity of parallel data leads to poor model performance, overfitting, and inadequate handling of named entities. While existing synthe...

Rabiu Ibrahim Abdullahi, G. Obunadike, A. Abdullahi et al. · 0 citations
Open access

Multilingual and cross-lingual machine reading comprehension in few-resource settings: synthetic data, cross-lingual transfer, and domain-specific evaluation

(English) Machine Reading Comprehension (MRC) is a fundamental benchmark for evaluating natural language understanding systems, requiring models to generate answers grounded in context. Despite advances with transformer-based architectures and large language models (LLMs), challenges persist in multilingual and cross-l...

C. Carrino · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.