MCMC Methods for Entropy Optimization and Nonlinear Network Coding

Abstract

Although determining the space of entropic vectors for n random variables, denoted by Γ^*_n, is crucial for solving a large class of network information theory problems, there has been scant progress in explicitly characterizing Γ^*_n for n ≥ 4. In this paper, we present a certain characterization of quasi-uniform distributions that allows one to numerically stake out the entropic region via a random walk to any desired accuracy. When coupled with Monte Carlo Markov Chain (MCMC) methods, one may "bias" the random walk so as to maximize certain functions of the entropy vector. As an example, we look at maximizing the violation of the Ingleton inequality for four random variables and report a violation well in excess of what has been previously available in the literature. Inspired by the MCMC method, we also propose a framework for designing optimal nonlinear network codes via performing a random walk over certain truth tables. We show that the method can be decentralized and demonstrate its efficacy by applying it to the Vamos network and a certain storage problem from [1].

Files

Additional details