A Caltech Library Service

The Role of Constraints in Hebbian Learning

Miller, Kenneth D. and MacKay, David J. C. (1994) The Role of Constraints in Hebbian Learning. Neural Computation, 6 (1). pp. 100-126. ISSN 0899-7667.

PDF - Published Version
See Usage Policy.


Use this Persistent URL to link to this item:


Models of unsupervised, correlation-based (Hebbian) synaptic plasticity are typically unstable: either all synapses grow until each reaches the maximum allowed strength, or all synapses decay to zero strength. A common method of avoiding these outcomes is to use a constraint that conserves or limits the total synaptic strength over a cell. We study the dynamic effects of such constraints. Two methods of enforcing a constraint are distinguished, multiplicative and subtractive. For otherwise linear learning rules, multiplicative enforcement of a constraint results in dynamics that converge to the principal eigenvector of the operator determining unconstrained synaptic development. Subtractive enforcement, in contrast, typically leads to a final state in which almost all synaptic strengths reach either the maximum or minimum allowed value. This final state is often dominated by weight configurations other than the principal eigenvector of the unconstrained operator. Multiplicative enforcement yields a “graded” receptive field in which most mutually correlated inputs are represented, whereas subtractive enforcement yields a receptive field that is “sharpened” to a subset of maximally correlated inputs. If two equivalent input populations (e.g., two eyes) innervate a common target, multiplicative enforcement prevents their segregation (ocular dominance segregation) when the two populations are weakly correlated; whereas subtractive enforcement allows segregation under these circumstances. These results may be used to understand constraints both over output cells and over input cells. A variety of rules that can implement constrained dynamics are discussed.

Item Type:Article
Related URLs:
URLURL TypeDescription
Additional Information:© 1994 Massachusetts Institute of Technology. Received October 9, 1992; accepted May 13, 1993. Posted Online April 4, 2008. K. D. M. thanks C. Koch, Caltech, and M. P. Stryker, UCSF, for supporting this work, which was performed in their laboratories. K. D. M. was supported by a Del Webb fellowship and a Markey Foundation internal grant, both from Caltech Division of Biology, and by an N.E.I. Fellowship at UCSF. D. J. C. M. was supported by a Caltech Fellowship and a Studentship from SERC, UK. We thank Bartlett Me1 and Terry Sejnowski for helpful comments on the manuscript. This collaboration would have been impossible without the internet/NSFnet.
Funding AgencyGrant Number
Del Webb fellowshipUNSPECIFIED
Markey Foundation internal grantUNSPECIFIED
Caltech FellowshipUNSPECIFIED
Studentship from SERC, UKUNSPECIFIED
Issue or Number:1
Record Number:CaltechAUTHORS:MILnc94
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:13800
Deposited By: Tony Diaz
Deposited On:25 Mar 2009 20:30
Last Modified:03 Oct 2019 00:44

Repository Staff Only: item control page