Prompting Beats Fine-Tuning: Generative Expected Value Scoring for Statutory Term Retrieval
TL;DR
This work studies the task of ranking case-law sentences by their usefulness for explaining a concept or target statutory term, using an established dataset of 26,959 sentences covering 42 U.S. Code concepts labeled into four explanatory-value categories.
Abstract
Legal concepts in statutes are often expressed using vague terms, and practitioners frequently turn to case law to interpret them. We study the task of ranking case-law sentences by their usefulness for explaining a concept or target statutory term, using an established dataset of 26,959 sentences covering 42 U.S. Code concepts labeled into four explanatory-value categories. We compare two families of methods: (i) supervised fine-tuning of encoder-only models (ModernBERT) and (ii) zero-shot prompting of decoder-only models. We show that across all concepts and standard NDCG cutoffs, ModernBERT largely matches earlier BERT-family baselines. In contrast, prompting decoder-only models achieves the strongest overall effectiveness, with our best system surpassing all previously reported state-of-the-art results on this task.