From Translation to Retrieval: Evaluating LLM-Based Information Retrieval for Hausa and Fongbe
TL;DR
A large LLM-capability gap between the two languages is confirmed, and data augmentation experiments across three encoder models show that LLM-generated text consistently hurts downstream NER tasks while producing mixed effects on POS tagging, motivating careful language-specific IR evaluation.
Abstract
LLM-based reranking has been evaluated for some African languages, but whether LLM-based query expansion helps or hurts retrieval for low-resource African languages remains an open question. Adeyemi et al. evaluated cross-lingual LLM reranking for Hausa with English queries, yet to our knowledge no published work has evaluated LLM-based query expansion for Hausa or Fongbe specifically, and no IR evaluation resources were found for Fongbe. This study builds upon our prior work on LLM translation quality evaluation and data augmentation for corpus expansion in Hausa and Fongbe. We propose experiments that compare LLM reranking and query expansion against BM25 and multilingual dense retrieval baselines (mDPR, mContriever) for Hausa and Fongbe using three commercial LLMs. Our completed translation quality assessment confirms a large LLM-capability gap between the two languages (best BLEU: Hausa 15.75 vs. Fongbe 7.18; human scores 4.5/5 vs. 2.2/5), and our data augmentation experiments across three encoder models show that LLM-generated text consistently hurts downstream NER tasks while producing mixed effects on POS tagging, motivating careful language-specific IR evaluation. We plan to use the CIRAL test collection for Hausa and to construct a new cross-lingual test set derived from Fongbe Wikipedia data following the AfriCLIRMatrix methodology.