Heterogeneous-Horizon Conformal Ensembles for Online Prediction Under Distribution Shift: An Empirical Comparison with Strongly Adaptive Methods

Abstract

Online conformal prediction methods such as Adaptive Conformal Inference (ACI) and Fully Adaptive Conformal Inference (FACI) adjust prediction intervals under distribution shift, but their calibration is based on a common stream of recent nonconformity scores. We introduce Population-based Adaptive Conformal Ensembles (PACE), a heuristic method that maintains online conformal quantile calibrators with different window sizes, decay rates, and quantile scales. PACE combines the best-calibrated members through fitness-weighted top-K averaging and periodically refreshes the population using clonal selection. For context, Strongly Adaptive Online Conformal Prediction (SAOCP) is a benchmark method that combines online calibration experts operating over different time intervals and provides a formal strongly adaptive regret guarantee. PACE is heuristic and does not provide an analogous regret or coverage guarantee. We evaluate the method on two synthetic datasets and three real-world time series. Against five adaptive conformal baselines, PACE achieves higher empirical coverage during extreme regimes. Compared with SAOCP, it generally obtains higher coverage by producing wider intervals, resulting in less favorable interval scores on most datasets.