Do large language models keep the hedge? Uncertainty preservation when large language models summarize aviation and maritime incident reports

Abstract

Safety-incident reports are written with deliberate epistemic caution: A reporter marks what was seen, what was inferred, and what remains unsure, and a dropped hedge can turn a tentative observation into an apparent cause. As large language models take over summarizing such reports for triage, indexing, and search, whether they preserve that caution becomes a property of the safety record itself. This study asks whether models keep a reporter's expressed uncertainty when they summarize, testing two hypotheses fixed before any summary was generated: that models systematically harden uncertainty by default and that an explicit instruction to preserve it overcorrects into softening. Six open-weight models summarized 42 real incident reports, 30 from the United States Aviation Safety Reporting System and 12 from the United Kingdom Marine Accident Investigation Branch, under a plain prompt, a preserve prompt, and a paraphrase of it. Uncertainty change was measured two ways: at the passage level, by a deterministic epistemic-marker lexicon, and at the claim level, by aligning those markers between source and summary, each validated against manual annotation and cross-checked against two independent language-model judges. Under plain summarization, all six models hardened the reporter's uncertainty (claim-level hardening 0.59–0.85), dropping explicit hedges almost entirely (retention about 0.08) while epistemic modal verbs survived far better. The preserve instruction reversed the pattern into over-softening: In a cluster-robust logistic regression, it carried an odds ratio of 0.12 (95% CI [0.03, 0.49], p = 0.003) and held once summary length was controlled, so the change reflects expressed certainty rather than compression. In practical terms, the instruction lowered the predicted probability of hardening from about 0.43 to 0.13 (Cohen's h = 0.69), and the effect held in mixed-effects models and under adverse imputation of the missing outputs. The pattern was directionally consistent across the tested models and both instruction wordings. Automated safety triage that relies on model summaries should measure epistemic fidelity directly and not treat a single preserve instruction as a solution.