Efficient Conditioning Why Pseudo Observation Batch Bayesian Optimization Works When It Does not

arXiv cs.LG Papers

Summary

This paper provides a unified theoretical framework for pseudo observation batch Bayesian optimization, proving that Gaussian processes produce distinct batch points and that common methods like Constant Liar and Kriging Believer are instances of a single conditioning mechanism. It introduces the Structural Diversity Diagnostic (SDD) for testing surrogate compatibility and validates predictions across multiple benchmark functions and hyperparameter tuning.

arXiv:2605.18819v1 Announce Type: new Abstract: Constant Liar (CL), Kriging Believer (KB), and fantasy models are widely used for batch selection in parallel Bayesian Optimization, yet a unified theory explaining their effectiveness and conditions under which they fail has been lacking. We identify efficient conditioning as the key surrogate property the ability to update predictions in closed form when data is augmented. We prove that Gaussian Processes satisfy this requirement, producing provably distinct batch points with separation of order l, and that this holds for any acquisition function monotonically non decreasing in posterior uncertainty (EI, UCB, PI), with qualitatively similar behavior for Thompson Sampling. We unify CL, KB, and fantasy models as instances of a single conditioning mechanism differing only in the lie value distribution, and draw quantitative connections to Local Penalization (LP) and qualitative connections to Determinantal Point Processes (DPPs). To disentangle model structure from optimizer randomness, we introduce the Structural Diversity Diagnostic (SDD), a reusable methodology for testing surrogate compatibility. Experiments on Hartmann6D, Ackley 8D, Levy10D, and SVM hyperparameter tuning validate all theoretical predictions: CL or KBs implicit penalty matches or outperforms explicit LP greedy conditioning achieves convergence on par with joint qEI efficient conditioning extends to Multiquadric RBF networks; and parametric surrogates produce degenerate batches even when fully retrained (random forests), while neural networks regain diversity only at 15x the wall clock cost of GP conditioning. Robustness is confirmed across multiple initial datasets and under observation noise.
Original Article

Similar Articles

Regularized Offline Policy Optimization with Posterior Hybrid Bayesian Belief

arXiv cs.AI

This paper introduces Posterior Hybrid Bayesian Belief (PhyB), a framework that reformulates the expectation in Bayesian RL as a convex combination over dynamics models, enabling efficient regularized offline policy optimization with bounded objective discrepancy and state-of-the-art performance.

When Determinants Are Not Enough: Private Rare Switching

arXiv cs.LG

This note presents a research moment where Codex helped find a new rare-switching rule for private linear bandits, using the generalized Rayleigh quotient to overcome the failure of determinant-based monotonicity due to Gaussian noise.