CaltechAUTHORS
  A Caltech Library Service

Redundant Information Neural Estimation

Kleinman, Michael and Achille, Alessandro and Soatto, Stefano and Kao, Jonathan C. (2021) Redundant Information Neural Estimation. Entropy, 23 (7). Art. No. 922. ISSN 1099-4300. PMCID PMC8304362. doi:10.3390/e23070922. https://resolver.caltech.edu/CaltechAUTHORS:20210803-170853656

[img] PDF - Published Version
Creative Commons Attribution.

1MB

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20210803-170853656

Abstract

We introduce the Redundant Information Neural Estimator (RINE), a method that allows efficient estimation for the component of information about a target variable that is common to a set of sources, known as the “redundant information”. We show that existing definitions of the redundant information can be recast in terms of an optimization over a family of functions. In contrast to previous information decompositions, which can only be evaluated for discrete variables over small alphabets, we show that optimizing over functions enables the approximation of the redundant information for high-dimensional and continuous predictors. We demonstrate this on high-dimensional image classification and motor-neuroscience tasks.


Item Type:Article
Related URLs:
URLURL TypeDescription
https://doi.org/10.3390/e23070922DOIArticle
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8304362PubMed CentralArticle
https://www.github.com/mjkleinman/RINERelated ItemCode
ORCID:
AuthorORCID
Kao, Jonathan C.0000-0002-9298-0143
Additional Information:© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Received: 11 May 2021 / Revised: 1 July 2021 / Accepted: 14 July 2021 / Published: 20 July 2021. We thank the reviewers for the helpful comments and suggestions. We thank Parthe Pandit and Hengjie Yang for helpful discussions. We thank Krishna Shenoy and Sergey Stavisky for permission to use the neural recording data sets from a delayed reach task. M.K. was supported by the National Sciences and Engineering Research Council (NSERC). J.C.K. was supported by an NSF CAREER Award (#1943467). This research was supported by a UCLA Computational Medicine Amazon Web Services Award. Author Contributions: Conceptualization, M.K.; methodology, M.K. and A.A.; software, M.K.; writing—original draft preparation, M.K.; writing—review and editing, M.K., A.A., S.S., J.C.K.; supervision, A.A., S.S., J.C.K. All authors have read and agreed to the published version of the manuscript. Data Availability Statement: Code is available at: www.github.com/mjkleinman/RINE. The authors declare no conflict of interest.
Funders:
Funding AgencyGrant Number
Natural Sciences and Engineering Research Council of Canada (NSERC)UNSPECIFIED
NSFIIS-1943467
Amazon Web ServicesUNSPECIFIED
Issue or Number:7
PubMed Central ID:PMC8304362
DOI:10.3390/e23070922
Record Number:CaltechAUTHORS:20210803-170853656
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20210803-170853656
Official Citation:Kleinman, M.; Achille, A.; Soatto, S.; Kao, J.C. Redundant Information Neural Estimation. Entropy 2021, 23, 922. https://doi.org/10.3390/e23070922
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:110134
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:04 Aug 2021 18:26
Last Modified:04 Aug 2021 18:26

Repository Staff Only: item control page