Aug 2026· Moratuwa Engineering Research Conference· pp. 503-508· 0 citations· 39 references
Abstract
Spell correction is still a challenging problem for many languages, especially low-resource languages (LRLs). While pre-trained language models (PLMs) have been employed for spell correction, there has been no proper comparison across PLMs. We present the first empirical study on the effectiveness of the three types of PLMs for spell correction across multiple languages, including low-resource languages. We show that even relatively small PLMs such as the 270M-parameter Gemma 3 and mBART50, when fine-tuned on a dataset of only 5k sentences, can outperform rule-based spell correctors, highlighting a practical pathway for building effective spell correction systems with limited data. We also present a case study with Sinhala to shed light on the plight of spell correction for LRLs.
Grammatical error correction (GEC) systems for learner English are usually trained as general-purpose models and rarely account for the writer's first language (L1), although L1 influences error patterns and may affect correction quality. To address this, this paper investigates whether lightweight L1 conditioning impr...
А. Т. Валиева, János Tóth· 2026 IEEE 4th Conference on...· 0 citations
Results show that BPE produces the most compact and morphologically aligned subword representations, while the modified Unigram LM achieved the best overall downstream performance across tasks, underscore that tokenizer choice and vocabulary design are critical determinants of language model efficiency and performance...
Mátyás Osváth, Matej Molnár, Roland Gunics et al.· International Conference on...· 1 citation
These results indicate that for multilingual models spanning typologically diverse scripts, to obtain maximum benefits, romanization should be treated as a core design choice applied at pretraining rather than a post hoc fix.
Mu-Ge Zhang, Aaron Jencks, Krishna Badikela et al.· 0 citations
This work investigates parameter-efficient approaches for transferring knowledge from resource-rich source languages to low-resource target languages on Whisper, and proposes Sequential Adapter Stacking, which places a trainable target-language adapter on top of a frozen source-language adapter.
Subword tokenization is a standard technique for pre-trained language models, mapping text into sequences of tokens from a fixed-size vocabulary. Despite its widespread use, the impact of tokenization algorithms and vocabulary sizes on downstream performance remains underexplored, particularly for low- and medium-resou...
J. Daðason, H. Loftsson· Proceedings of the Thirty-Fi...· 0 citations
This work tests whether a language's typological distance from the others predicts its ideal number of co-training languages, and finds no dependable rule: any apparent relationship rests on a couple of languages and does not hold up.
Saman Rahbar· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
