TurkLegalRAG: A Novel Dataset and Hybrid Retrieval Pipeline for Turkish Law

Abstract

Accessing reliable legal information is difficult in Turkish natural language processing, where large language models often cite the wrong legal basis or fabricate text, and where structured, domain-specific data is scarce. The main contribution of this study is TurkLegalRAG, a Turkish legal dataset built from scratch for retrieval-augmented question answering. An automated pipeline, run under responsible-scraping safeguards, collected the 106 most active laws in force as of June 2026 from the official legislation repository and segmented them, article by article, into 16,001 context-preserving chunks. Each chunk is enriched with document metadata that prevents global context loss and supports a deterministic, source-grounded citation mechanism. The dataset is completed by a leakage-controlled benchmark of 446 questions, verified to share no overlap with the training data, together with aligned training sets that fully fine-tune the retriever and reranker and adapt the generator with LoRA. To show the value of these resources, we build a RAG system on them: a hybrid BM25 and BGE-M3 retriever fused by reciprocal rank fusion, article-level deduplication and source capping for context diversity, cross-encoder reranking, and an abstain mechanism that refuses low-confidence queries. On the benchmark, the correct article is retrieved for 95 percent of questions and the citation rate reaches 94 percent.