Multiscale Invertible Generative Networks for High-Dimensional Bayesian Inference
We propose a Multiscale Invertible Generative Network (MsIGN) and associated training algorithm that leverages multiscale structure to solve high-dimensional Bayesian inference. To address the curse of dimensionality, MsIGN exploits the low-dimensional nature of the posterior, and generates samples from coarse to fine scale (low to high dimension) by iteratively upsampling and refining samples. MsIGN is trained in a multi-stage manner to minimize the Jeffreys divergence, which avoids mode dropping in high-dimensional cases. On two high-dimensional Bayesian inverse problems, we show superior performance of MsIGN over previous approaches in posterior approximation and multiple mode capture. On the natural image synthesis task, MsIGN achieves superior performance in bits-per-dimension over baseline models and yields great interpret-ability of its neurons in intermediate layers.
© 2021 by the author(s). We appreciate valuable comments and discussion from the reviewers. The work of Thomas Y. Hou was in part supported by NSF Grants DMS-1912654 and DMS-1907977, and Shumao Zhang was in part supported by NSF Grant DMS-1912654. The authors would also like to thank Ka Chun Lam and Xiaodong He for their helpful discussion and thank Lin Xiao, Qiang Liu and Bo Dai for their helpful suggestions.
Submitted - 2105.05489.pdf
Supplemental Material - zhang21z-supp.pdf
Published - zhang21z.pdf