CaltechAUTHORS
  A Caltech Library Service

On the Optimal Scaling of the Modified Metropolis-Hastings Algorithm

Zuev, K. M. and Beck, J. L. and Katafygiotis, L. S. (2011) On the Optimal Scaling of the Modified Metropolis-Hastings Algorithm. In: 11th International Conference on Applications of Statistics and Probability in Civil Engineering, August 1-4, 2011, ETH, Zurich, Switzerland. (Unpublished) http://resolver.caltech.edu/CaltechAUTHORS:20120831-110926649

[img]
Preview
PDF - Presentation
See Usage Policy.

118Kb

Use this Persistent URL to link to this item: http://resolver.caltech.edu/CaltechAUTHORS:20120831-110926649

Abstract

Estimation of small failure probabilities is one of the most important and challenging problems in reliability engineering. In cases of practical interest, the failure probability is given by a high-dimensional integral. Since multivariate integration suffers from the curse of dimensionality, the usual numerical methods are inapplicable. Over the past decade, the civil engineering research community has increasingly realized the potential of advanced simulation methods for treating reliability problems. The Subset Simulation method, introduced by Au & Beck (2001a), is considered to be one of the most robust advanced simulation techniques for solving high-dimensional nonlinear problems. The Modified Metropolis-Hastings (MMH) algorithm, a variation of the original Metropolis-Hastings algorithm (Metropolis et al. 1953, Hastings 1970), is used in Subset Simulation for sampling from conditional high-dimensional distributions. The efficiency and accuracy of Subset Simulation directly depends on the ergodic properties of the Markov chain generated by MMH, in other words, on how fast the chain explores the parameter space. The latter is determined by the choice of one-dimensional proposal distributions, making this choice very important. It was noticed in Au & Beck (2001a) that the performance of MMH is not sensitive to the type of the proposal PDFs, however, it strongly depends on the variance of proposal PDFs. Nevertheless, in almost all real-life applications, the scaling of proposal PDFs is still largely an art. The issue of optimal scaling was realized in the original paper by Metropolis (Metropolis et al. 1953). Gelman, Roberts, and Gilks (Gelman et al. 1996) have been the first authors to publish theoretical results about the optimal scaling of the original Metropolis-Hastings algorithm. They proved that for optimal sampling from a high-dimensional Gaussian distribution, the Metropolis-Hastings algorithm should be tuned to accept approximately 25% of the proposed moves only. This came as an unexpected and counter-intuitive result. Since then a lot of papers has been published on the optimal scaling of the original Metropolis-Hastings algorithm. In this paper, in the spirit of Gelman et al. (1996), we address the following question which is of high practical importance: what are the optimal one-dimensional Gaussian proposal PDFs for simulating a high-dimensional conditional Gaussian distribution using the MMH algorithm? We present a collection of observations on the optimal scaling of the Modified Metropolis-Hastings algorithm for different numerical examples, and develop an optimal scaling strategy for MMH when it is employed within Subset Simulation for estimating small failure probabilities.


Item Type:Conference or Workshop Item (Paper)
Related URLs:
URLURL TypeDescription
http://www-bcf.usc.edu/~kzuev/papers/OptimalScalingICASP.pdfAuthorUNSPECIFIED
Additional Information:This work was supported by the Hong Kong Research Grants Council, under grant HKUST 614008 and by the National Science Foundation, under award number EAR-0941374. This support is gratefully acknowledged. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the National Science Foundation.
Funders:
Funding AgencyGrant Number
Hong Kong Research Grants CouncilHKUST 614008
NSFEAR-0941374
Record Number:CaltechAUTHORS:20120831-110926649
Persistent URL:http://resolver.caltech.edu/CaltechAUTHORS:20120831-110926649
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:33768
Collection:CaltechAUTHORS
Deposited By: Carmen Nemer-Sirois
Deposited On:04 Sep 2012 20:00
Last Modified:26 Dec 2012 16:05

Repository Staff Only: item control page