A Caltech Library Service

How multiplicity determines entropy and the derivation of the maximum entropy principle for complex systems

Hanel, Rudolf and Thurner, Stefan and Gell-Mann, Murray (2014) How multiplicity determines entropy and the derivation of the maximum entropy principle for complex systems. Proceedings of the National Academy of Sciences of the United States of America, 111 (19). pp. 6905-6910. ISSN 0027-8424. PMCID PMC4024900. doi:10.1073/pnas.1406071111.

[img] PDF - Published Version
See Usage Policy.


Use this Persistent URL to link to this item:


The maximum entropy principle (MEP) is a method for obtaining the most likely distribution functions of observables from statistical systems by maximizing entropy under constraints. The MEP has found hundreds of applications in ergodic and Markovian systems in statistical mechanics, information theory, and statistics. For several decades there has been an ongoing controversy over whether the notion of the maximum entropy principle can be extended in a meaningful way to nonextensive, nonergodic, and complex statistical systems and processes. In this paper we start by reviewing how Boltzmann–Gibbs–Shannon entropy is related to multiplicities of independent random processes. We then show how the relaxation of independence naturally leads to the most general entropies that are compatible with the first three Shannon–Khinchin axioms, the Graphic-entropies. We demonstrate that the MEP is a perfectly consistent concept for nonergodic and complex statistical systems if their relative entropy can be factored into a generalized multiplicity and a constraint term. The problem of finding such a factorization reduces to finding an appropriate representation of relative entropy in a linear basis. In a particular example we show that path-dependent random processes with memory naturally require specific generalized entropies. The example is to our knowledge the first exact derivation of a generalized entropy from the microscopic properties of a path-dependent random process.

Item Type:Article
Related URLs:
URLURL TypeDescription CentralArticle
Additional Information:© 2014 National Academy of Sciences. Freely available online through the PNAS open access option. Contributed by Murray Gell-Mann, April 4, 2014 (sent for review January 30, 2014) R.H. and S.T. thank the Santa Fe Institute for hospitality. M.G.-M. acknowledges the generous support of Insight Venture Partners and the Bryan J. and June B. Zwan Foundation. Author contributions: R.H., S.T., and M.G.-M. designed research, performed research, contributed new reagents/analytic tools, and wrote the paper. The authors declare no conflict of interest.
Funding AgencyGrant Number
Insight Venture PartnersUNSPECIFIED
Bryan J. and June B. Zwan FoundationUNSPECIFIED
Subject Keywords:thermodynamics; out-of-equilibrium process; driven systems; random walk
Issue or Number:19
PubMed Central ID:PMC4024900
Record Number:CaltechAUTHORS:20150824-094800526
Persistent URL:
Official Citation:Rudolf Hanel, Stefan Thurner, and Murray Gell-Mann How multiplicity determines entropy and the derivation of the maximum entropy principle for complex systems PNAS 2014 111 (19) 6905-6910; published ahead of print April 29, 2014, doi:10.1073/pnas.1406071111
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:59836
Deposited By: George Porter
Deposited On:24 Aug 2015 17:47
Last Modified:10 Nov 2021 22:26

Repository Staff Only: item control page