CaltechAUTHORS
  A Caltech Library Service

CodNN – Robust Neural Networks From Coded Classification

Raviv, Netanel and Jain, Siddharth and Upadhyaya, Pulakesh and Bruck, Jehoshua and Jiang, Anxiao (Andrew) (2020) CodNN – Robust Neural Networks From Coded Classification. In: 2020 IEEE International Symposium on Information Theory (ISIT). IEEE , Piscataway, NJ, pp. 2688-2693. ISBN 978-1-7281-6432-8. https://resolver.caltech.edu/CaltechAUTHORS:20200427-091804171

[img] PDF - Submitted Version
See Usage Policy.

181Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20200427-091804171

Abstract

Deep Neural Networks (DNNs) are a revolutionary force in the ongoing information revolution, and yet their intrinsic properties remain a mystery. In particular, it is widely known that DNNs are highly sensitive to noise, whether adversarial or random. This poses a fundamental challenge for hardware implementations of DNNs, and for their deployment in critical applications such as autonomous driving.In this paper we construct robust DNNs via error correcting codes. By our approach, either the data or internal layers of the DNN are coded with error correcting codes, and successful computation under noise is guaranteed. Since DNNs can be seen as a layered concatenation of classification tasks, our research begins with the core task of classifying noisy coded inputs, and progresses towards robust DNNs.We focus on binary data and linear codes. Our main result is that the prevalent parity code can guarantee robustness for a large family of DNNs, which includes the recently popularized binarized neural networks. Further, we show that the coded classification problem has a deep connection to Fourier analysis of Boolean functions.In contrast to existing solutions in the literature, our results do not rely on altering the training process of the DNN, and provide mathematically rigorous guarantees rather than experimental evidence.


Item Type:Book Section
Related URLs:
URLURL TypeDescription
https://doi.org/10.1109/ISIT44484.2020.9174480DOIArticle
https://arxiv.org/abs/2004.10700arXivDiscussion Paper
https://resolver.caltech.edu/CaltechAUTHORS:20200427-091132325Related ItemTechnical Report
ORCID:
AuthorORCID
Raviv, Netanel0000-0002-1686-1994
Jain, Siddharth0000-0002-9164-6119
Upadhyaya, Pulakesh0000-0003-1054-1380
Bruck, Jehoshua0000-0001-8474-0812
Jiang, Anxiao (Andrew)0000-0002-0120-7930
Additional Information:© 2020 IEEE.
Record Number:CaltechAUTHORS:20200427-091804171
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20200427-091804171
Official Citation:N. Raviv, S. Jain, P. Upadhyaya, J. Bruck and A. A. Jiang, "CodNN – Robust Neural Networks From Coded Classification," 2020 IEEE International Symposium on Information Theory (ISIT), Los Angeles, CA, USA, 2020, pp. 2688-2693, doi: 10.1109/ISIT44484.2020.9174480
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:102792
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:27 Apr 2020 16:29
Last Modified:28 Aug 2020 17:20

Repository Staff Only: item control page