Skip to content
Log in
#small language model Open access

Translate, Search, or Answer: Cost-Aware Cross-Lingual Retrieval for Kazakh Small Language Models

Akylbek Maxutov Nūrali Medeu Vladimir Albrekht Danial Danenov H. Varol
Aug 2026 · Big Data and Cognitive Computing · 0 citations · 26 references

TL;DR

This work systematically compares zero-shot parametric generation, in-language retrieval, and cross-lingual (translate-then-retrieve) web search using three 4B-parameter SLMs in both reasoning and non-reasoning modes and proposes a training-free, self-aware router that uses majority voting over repeated self-verification decisions to determine when to search the web, and when to escalate to a more capable cloud model.

Abstract

Small Language Models (SLMs) enable efficient deployment, but their limited parameter count constrains factual knowledge, particularly in low-resource languages like Kazakh. Integrating live web search can address this limitation, though its effectiveness is difficult to measure due to sparse in-language web indices and answer leakage during benchmarking. In this study, we systematically compare zero-shot parametric generation, in-language retrieval, and cross-lingual (translate-then-retrieve) web search using three 4B-parameter SLMs in both reasoning and non-reasoning modes. To evaluate factuality without search-engine leakage, we introduce machine-translated Kazakh versions of the FreshQA and DefAn benchmarks, and use GPQA as a Google-proof adversarial control. We also assess robustness across three prompt complexities, from simple JSON constraints to adversarial warnings that instruct the model to treat potentially unreliable context with caution. Finally, we propose a training-free, self-aware router that uses majority voting over repeated self-verification decisions to determine when to answer parametrically, when to search the web, and when to escalate to a more capable cloud model. Our results show that cross-lingual retrieval substantially outperforms in-language search on global factuality tasks, nearly doubling accuracy on FreshQA, while direct in-language search remains preferable for localized cultural queries. The choice of retrieval language depends on the task and does not always favor English. Additionally, cross-lingual retrieval is not consistently superior, because the best option depends on where relevant information is indexed. Pareto analysis indicates that cross-lingual search is on or near the optimal accuracy–latency frontier, adding minimal overhead compared to direct search. The router identifies which query types warrant retrieval, and as a system it tracks or exceeds always-search accuracy while issuing fewer searches and approaching the always-cloud ceiling at a fraction of its cost; per-query discrimination within a task family is weaker, which we quantify explicitly. Overall, this work offers a framework for optimizing and accurately measuring cross-lingual RAG pipelines in low-resource settings.

DOI Semantic Scholar
Read PDF

Similar papers

Preprint Jul 2026

The Effect of Multi-Lingual and Keyword Adversarial Injection on LLM Relevance Judgment

Large language models (LLMs) are increasingly being used as automated judges for relevance evaluation in information retrieval, yet their robustness to adversarial manipulation remains insufficiently understood, particularly in multilingual settings. In this work, we investigate the impact of cross-lingual prompt injection attacks on LLM-based relevance judgments using TREC Deep Learning collections and two open-weight models under established prompting frameworks. We examine both instruction-based and content-based injection strategies in 8 languages spanning different resource levels. Our results demonstrate that multilingual query-based injections are highly effective in inflating relevance scores while simultaneously evading existing prompt-injection defenses. We further found that, although existing defense mechanisms can be modified to mitigate such attacks, these injections can be easily adapted to bypass them. These findings highlight a critical gap in current defense approaches and demonstrate that language generalization can act as an attack vector, underscoring the need for more robust and proactive evaluation frameworks for LLM-as-a-judge systems.

Nguyen-Thanh-Thao Vo, Duy Duong Tuong, Oleg Zendel et al. · 0 citations
Aug 2026

DocuMind: An Offline Multilingual Retrieval-Augmented Generation System for Document Question Answering Using Local Large Language Models

DocuMind is a fully offline, privacy-preserving, multilingual Document Question Answering system built on the Retrieval-Augmented Generation (RAG) architecture, enabling true cross-lingual retrieval without any translation step.

Swetha M S, Dr. Sripalreddy · 0 citations
Preprint Aug 2026

L3Cube-IndicQuest v2: A Large-Scale Multilingual Benchmark for Evaluating Factual Knowledge of Large Language Models Across Indic Languages

We present L3Cube-IndicQuest v2, a large-scale gold-standard multilingual question-answering benchmark for evaluating the India-specific factual knowledge of Large Language Models (LLMs). The benchmark comprises 3,471 curriculum-grounded English question--answer pairs spanning nine domains, curated from educational curricula, competitive examination materials, and domain-specific reference books. We introduce a practical hybrid construction strategy that combines context-grounded LLM-based question generation and validation with semantic deduplication and human verification, enabling scalable creation of benchmark data while preserving annotation quality. The benchmark is translated into 19 Indic languages, yielding a publicly released multilingual dataset of 69,420 question--answer pairs across 20 languages. We evaluate six LLMs under three protocols: LLM-as-a-judge and two deterministic lexical criteria, exact-substring and word-overlap matching. All three produce almost the same model ranking, showing that the results do not depend on the choice of judge. The frontier commercial model leads by a wide margin, and among open-weight models Gemma4 31B outperforms the Indic-specialised Sarvam 30B in every evaluated Indic language.

Rinit Jain, Tirthraj Mahajan, Advait Joshi et al. · 0 citations
Preprint Aug 2026

The Illusion of Cross-Lingual Safety in Low-Resource Languages

This work investigates cross-lingual safety transfer in four African languages, Twi, Hausa, Amharic, and Swahili, using LoDNA, a new safety dataset that pairs literal translations with culturally localized prompts to demonstrate superficial safety alignment.

Abigail Oppong, P SAM SAHIL, Tadesse Destaw Belay et al. · 0 citations
Jun 2026

Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian

The results show that Romanian incurs a 3 to 5 percentage point drop relative to English in prompt-only settings, that few-shot prompting provides marginal gains over zero-shot, and that QLoRA fine-tuning improves macro F1-Score by more than 22 percentage points in both languages while reducing the cross-lingual gap from 3.3 to 1.4pp.

Dragoș-Mitruț Vasile, Elena-Simona Apostol, Stefan-Adrian Toma et al. · 0 citations
Book Open access Jul 2026

Speak Beyond English: Multilingual Prompts Improve Query Classification in Small Language Models

This paper tests whether prompting the same (frozen) SLM in three typologically diverse languages and aggregating the outputs can improve classification without retraining or translation, and suggests that cross-lingual diversity rather than surface-level input variation drives the gain.

Pratyay Banerjee, Panthadeep Bhattacharjee, Angshuman Jana · 0 citations

Related blog posts

Microsoft Research Blog Aug 31, 2026

GigaPath-Flash and GigaTIME-Flash: Toward population-scale discovery with efficient pathology foundation models

What if pathology foundation models could do more with less? GigaPath-Flash and GigaTIME-Flash cut computational demands while maintaining strong performance, opening the door to larger studies and broader exploration. The post GigaPath-Flash and GigaTIME-Flash: Toward population-scale discovery with efficient pathology foundation models appeared first on Microsoft Research.