CaltechAUTHORS
  A Caltech Library Service

Discrete recurrent neural networks for grammatical inference

Zeng, Zheng and Goodman, Rodney M. and Smyth, Padhraic (1994) Discrete recurrent neural networks for grammatical inference. IEEE Transactions on Neural Networks, 5 (2). pp. 320-330. ISSN 1045-9227. https://resolver.caltech.edu/CaltechAUTHORS:20190315-142359688

[img] PDF - Published Version
See Usage Policy.

967Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20190315-142359688

Abstract

We describe a novel neural architecture for learning deterministic context-free grammars, or equivalently, deterministic pushdown automata. The unique feature of the proposed network is that it forms stable state representations during learning-previous work has shown that conventional analog recurrent networks can be inherently unstable in that they cannot retain their state memory for long input strings. We have recently introduced the discrete recurrent network architecture for learning finite-state automata. Here we extend this model to include a discrete external stack with discrete symbols. A composite error function is described to handle the different situations encountered in learning. The pseudo-gradient learning method (introduced in previous work) is in turn extended for the minimization of these error functions. Empirical trials validating the effectiveness of the pseudo-gradient learning method are presented, for networks both with and without an external stack. Experimental results show that the new networks are successful in learning some simple pushdown automata, though overfitting and non-convergent learning can also occur. Once learned, the internal representation of the network is provably stable; i.e., it classifies unseen strings of arbitrary length with 100% accuracy.


Item Type:Article
Related URLs:
URLURL TypeDescription
https://doi.org/10.1109/72.279194DOIArticle
Additional Information:© 1994 IEEE. Manuscript revised September 23, 1993. The research described in this paper was supported in part by ONR and ARPA under grant number N00014-92-J-1860. In addition this work was carried out in part by the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration.
Funders:
Funding AgencyGrant Number
Office of Naval Research (ONR)N00014-92-J-1860
Advanced Research Projects Agency (ARPA)UNSPECIFIED
NASA/JPL/CaltechUNSPECIFIED
Issue or Number:2
Record Number:CaltechAUTHORS:20190315-142359688
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20190315-142359688
Official Citation:Zheng Zeng, R. M. Goodman and P. Smyth, "Discrete recurrent neural networks for grammatical inference," in IEEE Transactions on Neural Networks, vol. 5, no. 2, pp. 320-330, March 1994. doi: 10.1109/72.279194
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:93889
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:15 Mar 2019 21:41
Last Modified:03 Oct 2019 20:58

Repository Staff Only: item control page