CaltechAUTHORS
  A Caltech Library Service

Provable Methods for Training Neural Networks with Sparse Connectivity

Sedghi, Hanie and Anandkumar, Anima (2014) Provable Methods for Training Neural Networks with Sparse Connectivity. In: 3rd International Conference on Learning Representations (ICLR 2015), 7-9 May 2015, San Diego, CA. https://resolver.caltech.edu/CaltechAUTHORS:20190401-162914714

[img] PDF - Accepted Version
See Usage Policy.

148Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20190401-162914714

Abstract

We provide novel guaranteed approaches for training feedforward neural networks with sparse connectivity. We leverage on the techniques developed previously for learning linear networks and show that they can also be effectively adopted to learn non-linear networks. We operate on the moments involving label and the score function of the input, and show that their factorization provably yields the weight matrix of the first layer of a deep network under mild conditions. In practice, the output of our method can be employed as effective initializers for gradient descent.


Item Type:Conference or Workshop Item (Poster)
Related URLs:
URLURL TypeDescription
http://arxiv.org/abs/1412.2693arXivDiscussion Paper
Additional Information:A. Anandkumar is supported in part by Microsoft Faculty Fellowship, NSF Career award CCF-1254106, NSF Award CCF-1219234, ARO YIP Award W911NF-13-1-0084 and ONR Award N00014-14-1-0665. H. Sedghi is supported by ONR Award N00014-14-1-0665.
Funders:
Funding AgencyGrant Number
Microsoft Faculty FellowshipUNSPECIFIED
NSFCCF-1254106
NSFCCF-1219234
Army Research Office (ARO)W911NF-13-1-0084
Office of Naval Research (ONR)N00014-14-1-0665
Subject Keywords:Deep feedforward networks, sparse connectivity, ℓ1-optimization, Stein’s lemma
Record Number:CaltechAUTHORS:20190401-162914714
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20190401-162914714
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:94343
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:03 Apr 2019 14:39
Last Modified:03 Oct 2019 21:03

Repository Staff Only: item control page