Maximum Entropy Relaxation for Graphical Model Selection given Inconsistent Statistics

Abstract

We develop a novel approach to approximate a specified collection of marginal distributions on subsets of variables by a globally consistent distribution on the entire collection of variables. In general, the specified marginal distributions may be inconsistent on overlapping subsets of variables. Our method is based on maximizing entropy over an exponential family of graphical models, subject to divergence constraints on small subsets of variables that enforce closeness to the specified marginals. The resulting optimization problem is convex, and can be solved efficiently using a primal-dual interiorpoint algorithm. Moreover, this framework leads naturally to a solution that is a sparse graphical model.