Enhancing Thai legal reasoning in large language models using group relative policy optimization

Abstract

The application of Large Language Models (LLMs) to Thai legal question answering (QA) presents significant opportunities but faces challenges, particularly with complex legal reasoning, accurate citation, and the high cost of advanced alignment techniques. This thesis addresses these issues through a two-fold contribution. Firstly, it details the development of NitiBench, a comprehensive benchmark specifically designed for Thai legal QA. NitiBench, comprising the NitiBench-CCL and NitiBench-Tax datasets, was instrumental in identifying critical performance gaps in existing Retrieval-Augmented Generation (RAG) systems, particularly in areas of citation accuracy and handling complex legal queries, even when perfect contextual information was provided. This highlighted a key bottleneck in the LLM's inherent reasoning capabilities. Building upon these findings, the second major contribution of this thesis is the introduction and evaluation of our proposed approach to improve LLMs for Thai legal Question Answering. We propose using Group-Relative Policy Optimization (GRPO) combined with efficient reward proxies. Our method leverages BGE-M3 embeddings for cost-effective answer quality rewarding using semantic similarity score, which significantly reducing computational costs compared to traditional LLM-based judges. Experiments conducted on NitiBench demonstrate that our approach achieves substantial improvements, including up to 90\% gains in citation F1-score and a 31\% increase in joint quality metrics over instruction-tuned baselines. Crucially, this method shows improved robustness in legal reasoning tasks and highlights the benefits for low-resource language and limited budget alignment. This research offers a practical pathway to developing more accurate, reliable, and resource-efficient LLMs for the Thai legal domain.