To optimize expensive black-box functions over networks, one of the most dominant frameworks is distributed Bayesian optimization (DBO), where local information can be exchanged among agents. However, DBO suffers from low evaluation efficiency due to limited data in the initial stage and the high cost of evaluations among multiple objectives. To tackle these obstacles, we propose a prior-informed multi-task collaborative Bayesian optimization (PMCBO) algorithm over networks. Concretely, PMCBO integrates expert prior knowledge about the location of optimum into the distributed multi-task Bayesian optimization framework to reduce the cost of evaluations. Meanwhile, PMCBO combines multi-task Bayesian optimization with a collaborative mechanism to improve the evaluation efficiency. Furthermore, we rigorously prove that the cumulative regret bound of PMCBO can achieve sub-linearly with high probability, where the acquisition functions employ expected improvement (EI) and upper-confidence bound (UCB) based on a Gaussian process surrogate. Finally, we implement various experiments to evaluate the effectiveness of PMCBO. The experimental results demonstrate that PMCBO can achieve state-of-the-art performance and benefit all clients based on diverse benchmarks and prior characteristics.
Youming Ge, Hai Jiang, Zhi-Hang Ji· Mathematics· 0 citations
Large language model (LLM)-driven multi-agent systems typically require multiple model invocations and complex coordination during inference, and their execution strategies directly affect system accuracy, latency, and computational cost. Parallel execution provides a means to improve inference-time efficiency. From the perspective of inference-time execution, this paper models parallelism in multi-agent systems as two distinct levels of decision processes: Replica Parallelism, which explores multiple complete solution paths at the task level, and Structural Parallelism, which enables concurrent execution within a single solution path through task decomposition. However, the roles of different forms of parallelism and their interrelationships still lack systematic study in terms of unified organization and coordination. We therefore propose TIPEX, a controllable execution framework that unifies these two levels of parallelism and coordinates their roles within the inference process under a unified execution semantics while supporting systematic combinations and analyses of different parallel strategies and parameter configurations. Systematic experiments on the GAIA benchmark demonstrate that inference-time parallelism can significantly improve accuracy and reduce end-to-end latency at the cost of increased token consumption. Further analysis shows that Replica and Structural Parallelism exhibit complementary effects across task complexities, with tasks of intermediate difficulty benefiting most from their coordination, while overly aggressive parallel strategies do not necessarily yield better performance.
Zihan Xu, Haolin Tian, Hai Jiang· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
