Summarising Regulations: an Empirical Study of Long-Document Summarisation Methods Under Extreme Compression

Regulatory documents are extreme in length, structurally heterogeneous, and linguistically technical, yet they must be summarised with high precision because omissions or distortions can materially change compliance meaning. We introduce ReguSum, a dataset of U.S. agency regulatory documents from the Securities and Exchange Commission (SEC) and the Internal Revenue Service (IRS) (2020–2024), paired with agency-provided abstracts that we treat as gold reference summaries. Compared to widely used long-document summarisation benchmarks, ReguSum operates in a long-input with extreme compression ratios exceeding 200:1 that stresses content selection and length budgeting under practical context limits.

Using ReguSum, we evaluate three families of methods under a unif ied pipeline: (i) internal input structuring with length control (wholedocument truncation, dynamic chunking, and section-aware hierarchical summarisation); (ii) seven retrieval-augmented summarisation variants that differ in query formulation and context construction; and (iii) clustering-based semantic chunking with HDBSCAN, evaluated with global versus document-specific parameterisation and combined with hierarchical decoding. Overall, retrieval-augmented variants do not surpass strong internal-structuring baselines in this setting, whereas semantic chunking is most effective when paired with hierarchical summarisation.