Automated Argument Mining and Entity Recognition in Sri Lankan Legal Judgments Using Large Language Models

Abstract

Access to legal information remains a significant challenge in low-resource jurisdictions, where judicial documents are often lengthy, unstructured, and difficult to interpret. This paper presents a unified framework for automated argument mining and named entity recognition in Sri Lankan legal judgments using Large Language Models (LLMs). The core contribution is a domain-specific argumentation scheme designed to capture legal reasoning, entities, and argument structures within judicial texts. The proposed approach integrates preprocessing, entity extraction, and argument mining into a single pipeline, enabling structured representation of legal discourse. The core argumentation components achieve a weighted F1 score of 0.980 and core legal named entities achieve a weighted F1 score of 0.953, based on a subset of 253 LLM-generated annotations. Across the complete 22-label scheme, evaluated using 1,335 expert verdicts covering 1,332 annotations and three missed annotations, the system achieves a macro-average F1 score of 0.729, with generic entity categories such as ORG and PERSON scoring lower due to over-extraction. Furthermore, legal expert evaluation demonstrates that the scheme effectively captures both explicit and implicit reasoning patterns while maintaining contextual relationships between entities and arguments. The results highlight the potential of structured LLM-driven approaches for improving legal text analysis in low-resource settings.