Hierarchical BART for Abstractive Multi-Document Summarisation: A Two-Stage Inference Pipeline with Semantic Evaluation

Abstract

Synthesising information scattered across multiple related documents into a single readable summary is fundamentally harder than summarising a single text. When documents originate from independent authors, the same event may be described through contrasting lenses, with divergent vocabulary, conflicting details, or complementary facts that no single source captures alone. Neural abstractive models—BART, T5, and PEGASUS—have greatly narrowed the gap to human performance on single-document tasks, yet their fixed 1,024-token encoder window becomes a hard ceiling when applied to multi-document clusters. For news corpora the average cluster runs to roughly 2,200 tokens; for scientific literature it reaches 4,800. Standard truncation therefore quietly throws away 54% and 79% of the source material, respectively—a coverage shortfall that current leaderboards largely obscure. Architectural remedies such as GraphSum, PRIMERA, and TALER partially address this by extending context capacity or re-aligning representations, but each demands substantial engineering overhead—extra GNN layers, specialised pre-training objectives, or delicate adversarial training loops. We introduce HierBART (Hierarchical BART), which sidesteps the need for any of this by treating the limitation as an inference-time problem rather than an architectural one. The approach is deliberately minimal: chunk the token stream into fixed-width overlapping windows, summarise each with a fine-tuned BART-BASE model (Stage 1), concatenate those local summaries into a fusion sequence, and re-decode it through the same model a second time (Stage 2). Because the pipeline uses only a standard fine-tuned checkpoint with unchanged weights, any future improvement to the backbone—quantisation, distillation, better fine-tuning data—automatically carries over. Benchmarked on Multi-News and Multi-XScience, HierBART reaches ROUGE-1 of 45.70 on Multi-News, outstripping TALER by 12.86 points, and 32.68 on Multi-XScience (+4.38). BERTScore F1 (roberta-large) stands at 80.14 and 84.44 respectively. A five-condition validation-set ablation traces 11.73 of the 22.21-point cumulative gain to the two structural decisions—window chunking (+7.17) and two-pass fusion (+4.56)—ruling out decoding artefacts as the driver. Blind human evaluation by five postgraduate NLP students rates HierBART at 7.90/10 for Fluency and 8.10/10 for Informativeness (Fleiss κ = 0.61), confirming that metric gains translate to perceptibly better summaries.