A neuro-symbolic clinical decision support system for chronic pain management using structured derivation traces, adaptive compute, and knowledge graph constraint checking: comparison with 30 physicians from pain-relevant specialties

Abstract

Chronic pain management requires complex multidisciplinary reasoning integrating pharmacological, interventional, rehabilitative, and psychosocial approaches across multiple guideline frameworks (International Association for the Study of Pain [IASP], American College of Rheumatology [ACR], European Alliance of Associations for Rheumatology [EULAR], American Academy of Neurology [AAN], American Society of Regional Anesthesia and Pain Medicine [ASRA], and Centers for Disease Control and Prevention [CDC]). Large language models (LLMs) show promise in clinical reasoning but lack any mechanism that records whether encoded guideline criteria were matched, are susceptible to hallucination, and cannot provide auditable decision chains—critical limitations where polypharmacy risks, opioid safety, and nuanced pain mechanism classification (nociplastic vs. nociceptive vs. neuropathic) demand the highest decision rigor. Some classical neuro-symbolic (NS) systems offer formal guarantees, yet such systems are heterogeneous and can lack contextual flexibility. We present PainNS-Prove, a guideline-provable neuro-symbolic clinical decision support system for chronic pain. We evaluate non-inferiority against 30 board-certified physicians from three pain-relevant specialties (10 anesthesiologists and algologists, 10 PM&R specialists, and 10 rheumatologists), all in active chronic pain practice across 30 expert-validated clinical vignettes. The five-stage pipeline integrates Claude Opus 4.5, GPT-4.1, and Gemini 2.5 Pro: (1) multi-LLM structured chain-of-thought with pain-specific prompting, (2) knowledge graph constraint checking against a Neo4j pain ontology (52,418 nodes; 203,672 edges), (3) structured derivation-trace generation with a Lean 4-inspired rule engine, (4) adaptive compute allocation, (5) screening of factual claims via multi-source triangulation, designed to prevent unsupported content from reaching the output. Statistical analyses included non-inferiority testing (δ = 5%), Fleiss’ κ (30 raters), Cohen’s κ (435 pairs), expert variability envelope, inter-specialty ANOVA, error typology with Fisher’s exact tests, ablation with McNemar’s tests, and counterfactual robustness analysis. PainNS-Prove achieved 90.0% accuracy (27/30; 95% CI: 74.4–96.5%), exceeding the expert panel mean of 78.0% (SD = 6.9%). The primary analysis was recomputed with a cluster-aware conservative method based on the 30 rater-level accuracies. The estimated marginal accuracy difference between PainNS-Prove and the expert benchmark was + 12.0% points, with a one-sided 95% lower confidence bound of − 0.7% points. Because this lower bound exceeded the pre-specified margin of − 5% points, non-inferiority was supported (one-sided p = 0.014). The two-sided 95% confidence interval for the difference was − 3.8 to + 19.0% points and includes zero, so superiority was not demonstrated ( p = 0.121) and is not claimed. In sensitivity analyses that modelled the shared vignette structure (vignette-level paired analysis, two-way rater and vignette variance decomposition, vignette-cluster bootstrap), the one-sided 95% lower bound ranged from − 0.8 to 0.0% points, and the non-inferiority conclusion was unchanged. Specialty means: algology 82.3% (SD = 5.0%), PM&R 77.0% (SD = 6.4%), rheumatology 74.7% (SD = 7.2%); inter-specialty ANOVA F(2,27) = 3.93, p = 0.032. The system fell within the expert variability envelope (64.3–91.7%) in 93.3% of cases. Human–human Fleiss’ κ = 0.52 (95% CI: 0.46–0.58); system–expert κ = 0.68 (0.59–0.77). Error typology: 70.7% of human errors were cognitive biases (anchoring 33.3%, availability 22.2%, premature closure 15.2%), 18.2% polypharmacy/opioid safety oversight, 11.1% guideline misapplication. PainNS-Prove produced no errors classified as cognitive bias and none classified as polypharmacy oversight; 3 errors in total (atypical presentation 66.7%, insufficient context 33.3%). Full pipeline outperformed LLM-only by + 16.7% points, which did not reach statistical significance (exact McNemar p = 0.063). Counterfactual stability: 90.0% vs. LLM-only 66.7% ( p = 0.016). PainNS-Prove was non-inferior to a panel of 30 physicians with active chronic pain practice at the pre-specified margin. Among the three incorrect system responses, none was classified by the two adjudicators as cognitive bias or as polypharmacy oversight. Because the number of system errors was small and the taxonomy may classify human errors and system errors differently, no comparative safety inference should be drawn from these counts. These findings come from a single set of 30 vignettes and require multicentre validation before any clinical use.