Context-Dependent FHIR Serialisation Strategies for Clinical LLM Deployment: A Multi-Layer Benchmark on UK Core Data
TL;DR
This work presents FHIRBench-UK, evaluating five large language models across six serialisation formats and three clinical tasks on 100 UK Core FHIR patient bundles, and recommends task-aware serialisation routing as a zero-cost quality intervention for NHS FHIR-based LLM deployments.
Abstract
The choice of FHIR-to-text serialisation format significantly impacts clinical LLM quality (Kruskal-Wallis H=163.86, p<10^-33, delta=0.24 on a 5-point scale), yet remains unstudied as a clinical deployment variable. We present FHIRBench-UK, evaluating five large language models across six serialisation formats and three clinical tasks on 100 UK Core FHIR patient bundles (18,000 scored prompts across clean and perturbed cohorts). Our findings converge with independent work on open-weight models (Pator, 2026). The optimal format is context-dependent: raw_json dominates for clinical QA, hybrid_adaptive for clinical reasoning, and structured_markdown for summarisation. In 58% of model-task-complexity scenarios, raw_json is suboptimal. Model capability moderates format sensitivity: Claude Sonnet 4.5 shows 0.10-point sensitivity versus Llama 3.3's 0.39, making adaptive serialisation most valuable for budget-constrained deployments using mid-tier models. All findings replicate under clinically realistic data perturbation. The study additionally confirms a complete ranking inversion between token-level F1 and clinical quality (rho=-0.90), replicating US findings across UK Core profiles. We recommend task-aware serialisation routing as a zero-cost quality intervention for NHS FHIR-based LLM deployments.