CaltechAUTHORS
  A Caltech Library Service

A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processes

Song, Jialin and Chen, Yuxin and Yue, Yisong (2019) A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processes. Proceedings of Machine Learning Research, 89 . pp. 3158-3167. ISSN 1938-7228. https://resolver.caltech.edu/CaltechAUTHORS:20190205-101951355

[img] PDF - Published Version
See Usage Policy.

729Kb
[img] PDF - Submitted Version
See Usage Policy.

852Kb
[img] PDF - Supplemental Material
See Usage Policy.

218Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20190205-101951355

Abstract

How can we efficiently gather information to optimize an unknown function, when presented with multiple, mutually dependent information sources with different costs? For example, when optimizing a physical system, intelligently trading off computer simulations and real-world tests can lead to significant savings. Existing multi-fidelity Bayesian optimization methods, such as multi-fidelity GP-UCB or Entropy Search-based approaches, either make simplistic assumptions on the interaction among different fidelities or use simple heuristics that lack theoretical guarantees. In this paper, we study multi-fidelity Bayesian optimization with complex structural dependencies among multiple outputs, and propose MF-MI-Greedy, a principled algorithmic framework for addressing this problem. In particular, we model different fidelities using additive Gaussian processes based on shared latent relationships with the target function. Then we use cost-sensitive mutual information gain for efficient Bayesian optimization. We propose a simple notion of regret which incorporates the varying cost of different fidelities, and prove that MF-MI-Greedy achieves low regret. We demonstrate the strong empirical performance of our algorithm on both synthetic and real-world datasets.


Item Type:Article
Related URLs:
URLURL TypeDescription
http://proceedings.mlr.press/v89/song19b.htmlPublisherArticle
https://arxiv.org/abs/1811.00755arXivDiscussion Paper
ORCID:
AuthorORCID
Yue, Yisong0000-0001-9127-1989
Additional Information:© 2019 by the author(s). This work was supported in part by NSF Award #1645832, Northrop Grumman, Bloomberg, Raytheon, PIMCO, and a Swiss NSF Early Mobility Postdoctoral Fellowship.
Funders:
Funding AgencyGrant Number
NSFCNS-1645832
Northrop GrummanUNSPECIFIED
Bloomberg Data ScienceUNSPECIFIED
Raytheon CompanyUNSPECIFIED
PIMCOUNSPECIFIED
Swiss National Science Foundation (SNSF)UNSPECIFIED
Record Number:CaltechAUTHORS:20190205-101951355
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20190205-101951355
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:92661
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:05 Feb 2019 19:03
Last Modified:05 Mar 2020 17:48

Repository Staff Only: item control page