From confidence to caution: conformal gating for cross-session motor imagery brain-computer interfaces.
Abstract
Objective
Erroneous motor imagery brain-computer interface (MI-BCI) commands can cause unintended device actions, yet standard classifiers provide no mechanism to withhold uncertain predictions. We evaluate whether conformal prediction (CP) can serve as a command gate under cross-session EEG shift, where exchangeability cannot be assumed, and how gating-policy choice should vary with application-specific risk tolerance. APPROACH We compare threshold-based CP (THR), regularised adaptive prediction sets (RAPS/RAPS-Opt), and Chow's selective prediction. Experiments use DeepConvNet as the base classifier on OpenBMI (54 subjects, 2-class) and BCI Competition IV Dataset 2a (BCIC IV-2a: 9 subjects, 4-class). Five chronological split conditions evaluate calibration sources with increasing match to the target session. Performance is assessed using decision rate, wrong-decisive rate, empirical coverage, F0.5, and a risk-weighted utility U(λ). MAIN
Results
Ungated cross-session classifiers produced wrong-decisive rates of 37.5% on OpenBMI and 35.5% on BCIC IV-2a. Across THR and RAPS, these rates were reduced to 9-16% on OpenBMI and 6-8% on BCIC IV-2a (OpenBMI: all pHolm < 10-9, BCIC IV-2a: pHolm = 0.012). On OpenBMI, increasing calibration-test match made conformal gates more conservative: for THR, decision rate fell from 0.55 to 0.41 and wrong-decisive rate from 0.16 to 0.11 (pHolm < 0.001). Calibtrain, requiring no labelled target-session data, achieved the highest THR F0.5 among cross-session strategies (0.53). Chow issued more commands but incurred higher wrong-decisive rates. Under U(λ), the preferred policy shifted from Chow at λ = 1 to THR for λ ≥ 2; a sensitivity analysis re-optimising Chow's threshold directly for U(λ) did not alter this qualitative ordering (THR above Chow-U for all λ ≥ 2, dz = 0.70-0.93).
Significance
This study reframes cross-session MI-BCI reliability as an application-dependent safety-decisiveness trade-off, providing a framework for selecting calibration sources and uncertainty-aware gating policies according to wrong-command tolerance under session shift.