Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published May 1, 1989 | public
Journal Article Open

Linear and logarithmic capacities in associative neural networks


A model of associate memory incorporating global linearity and pointwise nonlinearities in a state space of n-dimensional binary vectors is considered. Attention is focused on the ability to store a prescribed set of state vectors as attractors within the model. Within the framework of such associative nets, a specific strategy for information storage that utilizes the spectrum of a linear operator is considered in some detail. Comparisons are made between this spectral strategy and a prior scheme that utilizes the sum of Kronecker outer products of the prescribed set of state vectors, which are to function nominally as memories. The storage capacity of the spectral strategy is linear in n (the dimension of the state space under consideration), whereas an asymptotic result of n/4 log n holds for the storage capacity of the outer product scheme. Computer-simulated results show that the spectral strategy stores information more efficiently. The preprocessing costs incurred in the two algorithms are estimated, and recursive strategies are developed for their computation.

Additional Information

© Copyright 1989 IEEE. Reprinted with permission. Manuscript received March 20, 1985; revised January 20, 1988. This work was supported in part by the National Science Foundation under Grant EET-8709198. The material in this paper was partially presented at the Workshop on Neural Networks for Computing, Santa Barbara, CA, April 1985. We wish to thank our colleagues Profs. Y.S. Abu-Mostafa, N. Farhat, J.J. Hopfield, R.J. McEliece, and E.C. Posner, and Mr. J. Hong with whom we have had many illuminating discussions on this subject. Our thanks also go to Dr. R. Winslow who kindly made available to us an analytical proof demonstrating that the degenerate spectral scheme always enters a stable state, and to our reviewers who made several useful suggestions for improving the clarity of the presentation. Correction; IEEE Transactions on Information Theory 53(2):854 February 2007.


Files (1.2 MB)
Name Size Download all
1.1 MB Preview Download
50.5 kB Preview Download

Additional details

August 22, 2023
October 16, 2023