Sep 2026· Inquiry@Queen's Undergraduate Research Conference Proceedings· 0 citations
TL;DR
This project seeks to create the infrastructure needed to make legal citation in OpenJustice rigorous and transparent through various enhancements to the extraction and connection of case metadata, and fully integrating the resulting citation graph into the reranker, maximizing LLM reasoning capabilities.
Abstract
As AI enters the high-stakes domain of law, firmly grounding large language models in accurate legal information is essential. OpenJustice, a legal knowledge and research platform, gives users case-specific reasoning flows that are built on trustworthy precedent case citations. However, those citations are only truly as trustworthy as the systems that retrieve and resolve them. A citation that is outdated, lacks authority, or silently conflicts with another ruling not only weakens answers, but also risks misleading the recipients. This project seeks to create the infrastructure needed to make legal citation in OpenJustice rigorous and transparent through various enhancements to the extraction and connection of case metadata. This will be leveraged by the reranker, a module that filters and structures retrieved case information before passing the most relevant information to the model's reasoning stage.
The approach used LLMs themselves to process legal documents, identifying citation relationships that other methods such as regex cannot, including the treatment of each precedent case (followed, distinguished, not followed, etc.) and updates to their appeal histories (reversed, affirmed, quashed, etc.) as typed edges between case nodes. Together these relationships form a citation graph, a traversable web displaying how cases interact, that the reranker can draw on to weigh authority and surface conflicts automatically.
Testing at a scale of 5000 recently filed cases produced over 70000 citation edges spanning 26 distinct relationship types, with citation linking accuracy of 97.2% against independent legal data. Token costs were also validated at this scale, using a smaller, three times more cost-efficient language model with a drop in accuracy of only ~1%. Future goals include further improving linking accuracy through a continued focus on citations that predate the standardized citation format set out in 1999, and fully integrating the resulting citation graph into the reranker, maximizing LLM reasoning capabilities.
Faculty Supervisor: Xiaodan Zhu
Supporting Instructors: Chu Fei Luo, Kevin Yu
Supporting data, adapters, predictions and code for the article *Low-Cost LoRA Fine-Tuning of Small Language Models for Multi-Step Arithmetic Reasoning* by Jake O'Grady, Asena Isik Gürhan, Chee Fong Ting and Effirul Ramlan (University of Galway). We generated 20,000 GSM8K-derived arithmetic problems with step-by-step s...
O'Grady, Jake, Gürhan, Asena Isik, Chee, Fong Ting et al.· Zenodo (CERN European Organi...· 465 citations
The results are packaged in the Greenfield Startup Model (GSM), which explains the priority of startups to release the product as quickly as possible, and the need to shorten time-to-market, by speeding up the development through low-precision engineering activities.
Carmine Giardino, Nicolò Paternoster, M. Unterkalmsteiner et al.· IEEE Transactions on Softwar...· 178 citations· ⚡14
Software startup companies develop innovative, software-intensive products within limited timeframes and with few resources, searching for sustainable and scalable business models.
M. Unterkalmsteiner, P. Abrahamsson, Xiaofeng Wang et al.· e-Informatica Software Engin...· 157 citations· ⚡17
This study conducts a case survey study based on the secondary data of the major pivots happened in 49 software startups, and demonstrates that customer need pivot is the most common among all pivot types.
Sohaib Shahid Bajwa, Xiaofeng Wang, Anh Nguyen-Duc et al.· Empirical Software Engineeri...· 127 citations· ⚡15
The comparison of adopter and non-adopter sample reveals three potential adoption inhibitor, security, data privacy, and portability, which underlines the importance of the technical and security perspectives for research investigating the adoption of technology.
Nattakarn Phaphoom, Xiaofeng Wang, S. Samuel et al.· Journal of Systems and Softw...· 111 citations· ⚡8
Objective There is dearth of information on the timeliness of antenatal care (ANC) uptake. This study aimed to determine the timely ANC uptake by a medically trained provider (MTP) as per the World Health Organization (WHO) recommendations and the country guideline. Methods Cross-sectional survey was done with 2,731 wo...
B. Sarker, Musfikur Rahman, T. Rahman et al.· PLoS ONE· 89 citations· ⚡7
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 20, 2026
Senior Research Scientist Jim Glass discusses how his MIT-IBM Computing Research Lab collaboration connects perception and language, and addresses the shortcoming of current models.
MIT News · Artificial Intelligence· news.mit.eduNov 20, 2026
Enforcing physics-based constraints prevents AI models from generating unreliable predictions for complex nonlinear systems in weather, climate, and turbulent flows.
An MIT and Dana-Farber Institute-led team used large language models to examine nearly 4,000 studies and hundreds of code repositories, revealing gaps in the information needed to scrutinize and reuse published research.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.