CaltechAUTHORS
  A Caltech Library Service

Architecture Agnostic Neural Networks

Talukder, Sabera and Raghavan, Guruprasad and Yue, Yisong (2020) Architecture Agnostic Neural Networks. . (Unpublished) https://resolver.caltech.edu/CaltechAUTHORS:20210119-161636048

[img] PDF - Submitted Version
Creative Commons Attribution.

1245Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20210119-161636048

Abstract

In this paper, we explore an alternate method for synthesizing neural network architectures, inspired by the brain's stochastic synaptic pruning. During a person's lifetime, numerous distinct neuronal architectures are responsible for performing the same tasks. This indicates that biological neural networks are, to some degree, architecture agnostic. However, artificial networks rely on their fine-tuned weights and hand-crafted architectures for their remarkable performance. This contrast begs the question: Can we build artificial architecture agnostic neural networks? To ground this study we utilize sparse, binary neural networks that parallel the brain's circuits. Within this sparse, binary paradigm we sample many binary architectures to create families of architecture agnostic neural networks not trained via backpropagation. These high-performing network families share the same sparsity, distribution of binary weights, and succeed in both static and dynamic tasks. In summation, we create an architecture manifold search procedure to discover families or architecture agnostic neural networks.


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
http://arxiv.org/abs/2011.02712arXivDiscussion Paper
ORCID:
AuthorORCID
Yue, Yisong0000-0001-9127-1989
Additional Information:Attribution 4.0 International (CC BY 4.0).
Record Number:CaltechAUTHORS:20210119-161636048
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20210119-161636048
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:107568
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:20 Jan 2021 15:27
Last Modified:20 Jan 2021 15:27

Repository Staff Only: item control page