CaltechAUTHORS
  A Caltech Library Service

How Deep Are Deep Gaussian Processes?

Dunlop, Matthew M. and Girolami, Mark A. and Stuart, Andrew M. and Teckentrup, Aretha L. (2018) How Deep Are Deep Gaussian Processes? Journal of Machine Learning Research, 19 (54). pp. 1-46. ISSN 1533-7928. https://resolver.caltech.edu/CaltechAUTHORS:20181108-140320751

[img] PDF - Published Version
Creative Commons Attribution.

3640Kb
[img] PDF - Submitted Version
See Usage Policy.

1975Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20181108-140320751

Abstract

Recent research has shown the potential utility of deep Gaussian processes. These deep structures are probability distributions, designed through hierarchical construction, which are conditionally Gaussian. In this paper, the current published body of work is placed in a common framework and, through recursion, several classes of deep Gaussian processes are defined. The resulting samples generated from a deep Gaussian process have a Markovian structure with respect to the depth parameter, and the effective depth of the resulting process is interpreted in terms of the ergodicity, or non-ergodicity, of the resulting Markov chain. For the classes of deep Gaussian processes introduced, we provide results concerning their ergodicity and hence their effective depth. We also demonstrate how these processes may be used for inference; in particular we show how a Metropolis-within-Gibbs construction across the levels of the hierarchy can be used to derive sampling tools which are robust to the level of resolution used to represent the functions on a computer. For illustration, we consider the effect of ergodicity in some simple numerical examples.


Item Type:Article
Related URLs:
URLURL TypeDescription
http://www.jmlr.org/papers/v19/18-015.htmlPublisherArticle
https://arxiv.org/abs/1711.11280arXivDiscussion Paper
ORCID:
AuthorORCID
Dunlop, Matthew M.0000-0001-7718-3755
Additional Information:© 2018 Matthew M. Dunlop, Mark A. Girolami, Andrew M. Stuart and Aretha L. Teckentrup. License: CC-BY 4.0, see https://creativecommons.org/licenses/by/4.0/. Attribution requirements are provided at http://jmlr.org/papers/v19/18-015.html. Submitted 1/18; Revised 8/18; Published 9/18. MG is supported by EPSRC grants [EP/R034710/1, EP/R018413/1, EP/R004889/1, EP/P020720/1], an EPSRC Established Career Fellowship EP/J016934/3, a Royal Academy of Engineering Research Chair, and The Lloyds Register Foundation Programme on Data Centric Engineering. AMS is supported by AFOSR Grant FA9550-17-1-0185 and by US National Science Foundation (NSF) grant DMS 1818977. ALT is partially supported by The Alan Turing Institute under the EPSRC grant EP/N510129/1.
Funders:
Funding AgencyGrant Number
Engineering and Physical Sciences Research Council (EPSRC)EP/R034710/1
Engineering and Physical Sciences Research Council (EPSRC)EP/R018413/1
Engineering and Physical Sciences Research Council (EPSRC)EP/R004889/1
Engineering and Physical Sciences Research Council (EPSRC)EP/P020720/1
Engineering and Physical Sciences Research Council (EPSRC)EP/J016934/3
Royal Academy of EngineeringUNSPECIFIED
Lloyds Register FoundationUNSPECIFIED
Air Force Office of Scientific Research (AFOSR)FA9550-17-1-0185
NSFDMS-1818977
Alan Turing InstituteUNSPECIFIED
Engineering and Physical Sciences Research Council (EPSRC)EP/N510129/1
Subject Keywords:deep learning, deep Gaussian processes, deep kernels
Issue or Number:54
Record Number:CaltechAUTHORS:20181108-140320751
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20181108-140320751
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:90763
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:09 Nov 2018 13:00
Last Modified:03 Oct 2019 20:28

Repository Staff Only: item control page