Skip to content
Preprint

Adaptive mixture variational inference for spike-and-slab regression

Sep 2026 · 0 citations · 20 references
Mathematics

TL;DR

An adaptive fitting procedure for mixtures of product distributions in Gaussian regression with a point-mass spike-and-slab prior that minimizes reverse Kullback-Leibler divergence directly on inclusion indicators and active coefficients, jointly refining component parameters and weights as the mixture grows.

Abstract

Correlated predictors can support competing sparse explanations with similar predictions, making joint uncertainty about variable inclusion difficult to capture with mean-field approximations. We develop an adaptive fitting procedure for mixtures of product distributions in Gaussian regression with a point-mass spike-and-slab prior. It minimizes reverse Kullback-Leibler divergence directly on inclusion indicators and active coefficients, jointly refining component parameters and weights as the mixture grows. This avoids an additional divergence penalty on unused latent coefficients under independent augmentation. Our analysis relates approximation accuracy to mixture size, support coverage and dependence within supports, and establishes contraction, selection consistency and a Bernstein-von Mises approximation under explicit conditions on the prior, posterior concentration and variational error. On all 250 simulated datasets with exact posterior references, mixtures reduce errors in inclusion probabilities, grouped support probabilities and coefficient covariance relative to multistart mean field. Comparisons at fixed mixture size and common initialization favor direct joint refinement over augmented or restricted refinement in posterior divergence. Complete stagewise fitting can nevertheless be more accurate near collinearity. The results support direct joint refinement for posterior approximation while showing that local gains do not ensure superiority of the full adaptive search.

View source

Similar papers

Open access Sep 2026

Structured Spike-and-Slab Variational Bayes for High-Dimensional Non-Normal Generalized Linear Mixed Models

High-dimensional non-normal longitudinal data are ubiquitous across fields such as genomics, biomedicine, microbiome research, and the social sciences. Such data often combine non-normal responses, within-subject dependence and sparse population effects. We develop a structured variational Bayesian procedure: variation...

Jie-Yi Yi, Ying Wu, Yun-Qi Zhang · 0 citations
Preprint Sep 2026

Proximal Empirical Bayes for Sparse Regression with Posterior Decision Support

In this work, we revisit estimation and variable selection in sparse Gaussian regression, possibly under affine constraints. We adopt a middle-ground approach between optimisation and fully hierarchical Bayesian methods, preserving much of the computational efficiency of the former while adding posterior uncertainty qu...

Dimitrios Roxanas · 0 citations
#machine learning Preprint Oct 2026

LinearPFN: Amortized Variable Selection for Linear Models with Interactions

Spike-and-slab regression is a standard Bayesian formulation of variable selection: it returns a posterior distribution over which candidate effects are active rather than a single selected subset, so that every candidate effect carries an inclusion probability. Its cost grows exponentially with the number of candidate...

Louis Schiekiera, Max Zimmer, Christophe Roux et al. · 1 citation
#machine learning Preprint Oct 2026

Latent Annealed Posterior Sampling with a Variance-Preserving Bridge for Sparsely Observed Random Fields

Posterior inference for random fields from sparse observations presents two major challenges: accurately characterizing complex spatial uncertainty and preventing conditional sampling costs from growing substantially as the spatial grid is refined. To address these challenges, we propose latent annealed posterior sampl...

Jian-Kun Lei, Tian-Xi Qi · 0 citations
Preprint Sep 2026

Robust and Sparse Group Dynamic Causal Modeling via Student-t Parametric Empirical Bayes and Nonlocal Priors

Dynamic causal modeling (DCM) estimates directed effective connectivity, while parametric empirical Bayes (PEB) supports group inference using subject-specific posterior summaries. Standard PEB relies on Gaussian models and continuous shrinkage, making it sensitive to atypical estimates and unable to distinguish neglig...

G. Arhin, Nilotpal Sanyal · 0 citations
#artificial intelligence Preprint Oct 2026

Posterior sampling by source-space MCMC via prior-based few-step transport maps

This work develops a framework for source-space generalized Bayesian inference that combines inexpensive few-step prior transports with posterior stability guarantees and establishes Wasserstein error bounds between the exact and learned posteriors in terms of the joint population iMF and auxiliary-velocity loss.

Hoang Phuc Hau Luu, Marcelo Hartmann, Zhong-Jian Wang · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.