Bidirectional Information Flow (BIF) - A Sample Efficient Hierarchical Gaussian Process for Bayesian Optimization
TL;DR
B Bidirectional Information Flow is proposed, which establishes continuous two-way communication in hierarchical Gaussian Process models and proves analytically that the regret of a GP with a learned kernel scales linearly with the mismatch to the true kernel.
Abstract
Hierarchical Gaussian Process (H-GP) models divide problems into different subtasks, allowing different components to address each part, making them well-suited for problems with inherent compositional structure. However, existing H-GP frameworks typically employ one-way information sharing - either top-down or bottom-up - which limits sample efficiency and slows convergence. We propose Bidirectional Information Flow (BIF), which establishes continuous two-way communication. BIF retains the modular structure of hierarchical models-the parent conditions its own posterior on child summaries, treating them as structured priors-while introducing top-down feedback to softly decompose environment observations from the parent into sub-responses. This mutual exchange improves sample efficiency, enables robust training, and allows modular reuse of learned subtask models. We prove analytically that the regret of a GP with a learned kernel scales linearly with the mismatch to the true kernel, tightening in the hierarchical case to the sum of child-level errors. Ablation shows that removing the downward pathway collapses child $R^2$ by up to 58%. Across synthetic, neurostimulation, and HPO benchmarks, BIF achieves up to $4\times$ higher parent $R^2$ and $\sim 100\%$ AUC improvement over vanilla GPBO, and outscores all hierarchical state-of-the-art methods on child $R^2$ given the correct acquisition function, while supporting modular child transfer to novel composite tasks.
