A Caltech Library Service

Consistency of Empirical Bayes And Kernel Flow For Hierarchical Parameter Estimation

Chen, Yifang and Owhadi, Houman and Stuart, Andrew M. (2020) Consistency of Empirical Bayes And Kernel Flow For Hierarchical Parameter Estimation. . (Unpublished)

[img] PDF - Submitted Version
See Usage Policy.


Use this Persistent URL to link to this item:


Hierarchical modeling and learning has proven very powerful in the field of Gaussian process regression and kernel methods, especially for machine learning applications and, increasingly, within the field of inverse problems more generally. The classical approach to learning hierarchical information is through Bayesian formulations of the problem, implying a posterior distribution on the hierarchical parameters or, in the case of empirical Bayes, providing an optimization criterion for them. Recent developments in the machine learning literature have suggested new criteria for hierarchical learning, based on approximation theoretic considerations that can be interpreted as variants of cross-validation, and exploiting approximation consistency in data splitting. The purpose of this paper is to compare the empirical Bayesian and approximation theoretic approaches to hierarchical learning, in terms of large data consistency, variance of estimators, robustness of the estimators to model misspecification, and computational cost. Our analysis is rooted in the setting of Matérn-like Gaussian random field priors, with smoothness, amplitude and inverse lengthscale as hierarchical parameters, in the regression setting. Numerical experiments validate the theory and extend the scope of the paper beyond the Matérn setting.

Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription Paper
Owhadi, Houman0000-0002-5677-1600
Additional Information:YC gratefully acknowledges the support of the Caltech Kortchack Scholar Program. HO gratefully acknowledges support from AFOSR (grant FA9550-18-1-0271) and ONR (grant N00014-18-1-2363). AMS is grateful to AFOSR (grant FA9550-17-1-0185) and NSF (grant DMS 18189770) for financial support.
Funding AgencyGrant Number
Kortschak Scholars ProgramUNSPECIFIED
Air Force Office of Scientific Research (AFOSR)FA9550-18-1-0271
Office of Naval Research (ONR)N00014-18-1-2363
Air Force Office of Scientific Research (AFOSR)FA9550-17-1-0185
Classification Code:2010 Mathematics Subject Classification. 65F12, 62C10, 41A05, 35Q62
Record Number:CaltechAUTHORS:20201109-141002843
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:106560
Deposited By: George Porter
Deposited On:09 Nov 2020 22:45
Last Modified:09 Nov 2020 22:45

Repository Staff Only: item control page