CaltechAUTHORS
  A Caltech Library Service

Memory Augmented Recursive Neural Networks

Arabshahi, Forough and Lu, Zhichu and Singh, Sameer and Anandkumar, Animashree (2019) Memory Augmented Recursive Neural Networks. . (Unpublished) https://resolver.caltech.edu/CaltechAUTHORS:20200109-090330653

[img] PDF - Submitted Version
See Usage Policy.

869kB

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20200109-090330653

Abstract

Recursive neural networks have shown an impressive performance for modeling compositional data compared to their recurrent counterparts. Although recursive neural networks are better at capturing long range dependencies, their generalization performance starts to decay as the test data becomes more compositional and potentially deeper than the training data. In this paper, we present memory-augmented recursive neural networks to address this generalization performance loss on deeper data points. We augment Tree-LSTMs with an external memory, namely neural stacks. We define soft push and pop operations for filling and emptying the memory to ensure that the networks remain end-to-end differentiable. In order to assess the effectiveness of the external memory, we evaluate our model on a neural programming task introduced in the literature called equation verification. Our results indicate that augmenting recursive neural networks with external memory consistently improves the generalization performance on deeper data points compared to the state-of-the-art Tree-LSTM by up to 10%.


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
http://arxiv.org/abs/1911.01545arXivDiscussion Paper
Additional Information:A. Anandkumar is supported by Bren Chair professorship, DARPA PAI HR0011-18-9-0035, Faculty awards from Adobe, BMW, Microsoft and Google.
Funders:
Funding AgencyGrant Number
Bren Professor of Computing and Mathematical SciencesUNSPECIFIED
Defense Advanced Research Projects Agency (DARPA)HR0011-18-9-0035
AdobeUNSPECIFIED
BMWUNSPECIFIED
MicrosoftUNSPECIFIED
GoogleUNSPECIFIED
Record Number:CaltechAUTHORS:20200109-090330653
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20200109-090330653
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:100579
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:09 Jan 2020 19:40
Last Modified:09 Jan 2020 19:40

Repository Staff Only: item control page