Hierarchical Classification of Arabic Legal Cases Using Transformer Architectures and Large Language Models
TL;DR
The experimental results demonstrate that the Hierarchy-Aware architecture consistently achieves the strongest fine-grained classification performance across both model families, highlighting the importance of explicit hierarchical supervision and domain-adapted pre-training for Arabic legal text classification, and establish strong baselines for future research in this underexplored domain.
Abstract
Automated classification of Arabic legal texts presents unique challenges stemming from the formal register of judicial language, domain-specific Sharī‘a terminology, and the severe class imbalance inherent in hierarchical legal taxonomies. This paper addresses these challenges through a systematic investigation of hierarchical multi-class classification applied to a dataset of 1146 Arabic judicial cases sourced from the Saudi Ministry of Justice open data portal. Cases are annotated at two hierarchical levels: a binary main-class distinction between criminal and civil cases (L1), and a fine-grained sub-category classification across 63 legal topics (L2), exhibiting a class imbalance ratio of 43:1. We evaluate three methodological families under four classification architectures. For encoder-only models, we fine-tune AraBERT, CAMeL-BERT, MARBERTv2, and XLM-RoBERTa under Flat (Bottom-Up), Hierarchical Multi-Task, Two-Stage, and Hierarchy-Aware configurations. For encoder–decoder models, we adapt AraT5v2 under the same four architectures. Finally, we conduct a prompt-based evaluation of GPT-4o under zero-shot and targeted few-shot settings. The experimental results demonstrate that the Hierarchy-Aware architecture consistently achieves the strongest fine-grained classification performance across both model families. AraBERTv2 with Hierarchy-Aware training achieves the best overall L2 accuracy of 83.04% and a Macro-F1 of 76.07%, while the Single Multi-Task configuration achieves the highest L1 accuracy of 99.57%. GPT-4o under 5-shot prompting achieves 99.13% L1 accuracy and a competitive L2 Macro-F1 of 74.30% without task-specific fine-tuning, though supervised models maintain stronger overall fine-grained performance. These findings highlight the importance of explicit hierarchical supervision and domain-adapted pre-training for Arabic legal text classification, and establish strong baselines for future research in this underexplored domain.