A Caltech Library Service

A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processes

Song, Jialin and Chen, Yuxin and Yue, Yisong (2019) A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processes. Proceedings of Machine Learning Research, 89 . pp. 3158-3167. ISSN 1938-7228.

[img] PDF - Published Version
See Usage Policy.

[img] PDF - Submitted Version
See Usage Policy.

[img] PDF - Supplemental Material
See Usage Policy.


Use this Persistent URL to link to this item:


How can we efficiently gather information to optimize an unknown function, when presented with multiple, mutually dependent information sources with different costs? For example, when optimizing a physical system, intelligently trading off computer simulations and real-world tests can lead to significant savings. Existing multi-fidelity Bayesian optimization methods, such as multi-fidelity GP-UCB or Entropy Search-based approaches, either make simplistic assumptions on the interaction among different fidelities or use simple heuristics that lack theoretical guarantees. In this paper, we study multi-fidelity Bayesian optimization with complex structural dependencies among multiple outputs, and propose MF-MI-Greedy, a principled algorithmic framework for addressing this problem. In particular, we model different fidelities using additive Gaussian processes based on shared latent relationships with the target function. Then we use cost-sensitive mutual information gain for efficient Bayesian optimization. We propose a simple notion of regret which incorporates the varying cost of different fidelities, and prove that MF-MI-Greedy achieves low regret. We demonstrate the strong empirical performance of our algorithm on both synthetic and real-world datasets.

Item Type:Article
Related URLs:
URLURL TypeDescription Paper
Yue, Yisong0000-0001-9127-1989
Additional Information:© 2019 by the author(s). This work was supported in part by NSF Award #1645832, Northrop Grumman, Bloomberg, Raytheon, PIMCO, and a Swiss NSF Early Mobility Postdoctoral Fellowship.
Funding AgencyGrant Number
Northrop GrummanUNSPECIFIED
Bloomberg Data ScienceUNSPECIFIED
Raytheon CompanyUNSPECIFIED
Swiss National Science Foundation (SNSF)UNSPECIFIED
Record Number:CaltechAUTHORS:20190205-101951355
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:92661
Deposited By: Tony Diaz
Deposited On:05 Feb 2019 19:03
Last Modified:05 Mar 2020 17:48

Repository Staff Only: item control page