Equal Budgets Change the Verdict: Finite-Sample Bias and a Matched-Budget Re-Examination of Diversity-Enhanced flowMC Ensembles

Abstract

Normalizing-flow Markov chain Monte Carlo (MCMC), such as flowMC, augments local moves with a learned global flow proposal; a natural reliability idea is to pool samples from several such samplers. On 6 targets in 20 and 50 dimensions, a 5-member diverse flowMC ensemble appeared to reduce the average marginal Jensen–Shannon (JS) distance by about 18% relative to a single flowMC run. Matching the returned sample counts—our “equal budget”: wall-clock costs differ and are reported separately—reverses this verdict: the ensemble draws five times as many samples, and even a perfect sampler’s histogram JS estimate has a closed root-mean-square finite-sample floor c(1/N+1/M), with c a fitted coefficient set by the binning, which explains the apparent gain almost entirely. At a matched 10,000-sample budget, a single flowMC run matches or beats this ensemble on all eleven configurations and runs about six times faster. Applied to exact draws, the uncorrected reweight-and-resample aggregation reproduces about 93% of the ensemble’s elevation above the floor, separating a genuine law shift (coverage and tempering reweighting) from an estimator-level loss (resampling). On two real Bayesian posteriors scored against long NUTS references, uniform pooling improved on the sample-count-matched single run under marginal JS, energy distance, and MMD2 in every recorded run (a descriptive comparison at three and two repetitions), while the aggregation’s deficit is confined to the biased marginal metric. We recommend matched sample counts and floor reporting for histogram divergences, unbiased joint metrics alongside, and uniform pooling instead of uncorrected reweighting and resampling.