Towards Accessible Legal AI: A Hierarchical Transformer-Based Approach for Summarizing Long Indian Legal and Policy Documents

Abstract

Policy and governmental documents tend to be quite extensive and contain complex legal language, which makes them less accessible to people. The existing transformer-based summarization approaches are intended for short documents and face difficulties in processing long legal and governmental documents in a readable manner. Summarization techniques, such as BART, PEGASUS, and T5, have achieved great success in abstractive summarization, but their input sequence lengths' limitations and lack of hierarchical modeling make their application to legal and long governmental documents impossible. This paper presents SET (Summarization for Enhanced Transparency), which is a hierarchical transformer-based approach to summarizing Indian legal and governmental documents that includes, sentence selection based on TF-IDF, abstractive summarization at the level of chunks based on BART-large-CNN, deduplication and filtering and the accessibility module for simplifying legal language. The hierarchical architecture allows performing the processing on CPUs and summarization of the documents with up to 200,000 words in length by splitting into chunks. Tested on a curated dataset with 77 documents of legal and governmental domain $(\approx 2.78 \mathrm{M}$ words), SET demonstrates the ROUGE-2 score of 0.102, which surpasses baselines, while improving readability from Flesch-Kincaid grade of 15-25 to 50-60.