assistcurve: decision-analytic evaluation of clinician-plus-AI decisions

Abstract

First public release of assistcurve, the software accompanying the manuscript Assistance curve analysis: evaluating the clinical value of AI-assisted decision making. What it does Assistance curve analysis extends decision curve analysis from prediction models to a clinician working with a model. The central quantity is the net benefit of assistance, ΔNB(p): the difference in net benefit between the decisions a clinician makes with an AI available and the decisions they make without it. Because net benefit is linear in the decisions, ΔNB depends only on the cases where the clinician changed their mind, and reduces to four numbers: ΔNB(p) = [ r_U(π_U − p) + r_L(p − π_L) ] / (1 − p) which gives a break-even rule anyone can apply: follow the AI upward only if more than p of the patients it talks you into treating really have the condition, and downward only if fewer than p of those it talks you out of treating do. The package provides the assistance curve, the switch decomposition, the break-even threshold, the number needed to consult, and an override profile that identifies which part of a model's output range earned its place on the screen. Contents assistcurve/ — the Python package R/assistcurve/ — the R package, with documentation and its own test suite analysis/ — the simulation study, the Collab-CXR analysis, the permutation test, and verify_reported_values.py data/ — derived analysis files figures/ — the six manuscript figures, light and dark Reproducibility 18 Python tests and 22 R tests over 88 expectations. Both suites check properties — the two theorems, the affine form of the curve, the additivity of the override bands — rather than stored output. analysis/verify_reported_values.py recomputes all 144 values reported in the manuscript and exits non-zero on any disagreement. CI runs both suites on Python 3.9/3.11/3.13 and R 3.6/release, verifies the published values, and asserts that the committed derived tables match a fresh run. The R and Python implementations agree to ~1e-16 on every deterministic quantity. Data The applied analysis uses Collab-CXR (Moehring et al., CC-BY 4.0), https://doi.org/10.17605/OSF.IO/Z7APQ. The 36 MB raw file is not redistributed here; data/ holds the derived files, which reproduce every design 2 and design 3 number and every real-data figure. See analysis/HOW_TO_RERUN.md. Licence MIT.