CaltechAUTHORS
  A Caltech Library Service

Temporal evolution of generalization during learning in linear networks

Baldi, Pierre and Chauvin, Yves (1991) Temporal evolution of generalization during learning in linear networks. Neural Computation, 3 (4). pp. 589-603. ISSN 0899-7667. http://resolver.caltech.edu/CaltechAUTHORS:BALnc91b

[img]
Preview
PDF - Published Version
See Usage Policy.

643Kb

Use this Persistent URL to link to this item: http://resolver.caltech.edu/CaltechAUTHORS:BALnc91b

Abstract

We study generalization in a simple framework of feedforward linear networks with n inputs and n outputs, trained from examples by gradient descent on the usual quadratic error function. We derive analytical results on the behavior of the validation function corresponding to the LMS error function calculated on a set of validation patterns. We show that the behavior of the validation function depends critically on the initial conditions and on the characteristics of the noise. Under certain simple assumptions, if the initial weights are sufficiently small, the validation function has a unique minimum corresponding to an optimal stopping time for training for which simple bounds can be calculated. There exists also situations where the validation function can have more complicated and somewhat unexpected behavior such as multiple local minima (at most n) of variable depth and long but finite plateau effects. Additional results and possible extensions are briefly discussed.


Item Type:Article
Related URLs:
URLURL TypeDescription
http://dx.doi.org/10.1162/neco.1991.3.4.589DOIUNSPECIFIED
http://www.mitpressjournals.org/doi/abs/10.1162/neco.1991.3.4.589PublisherUNSPECIFIED
Additional Information:© 1991 Massachusetts Institute of Technology. Received 1 February 1991; accepted 13 April 1991. Posted Online March 13, 2008. This work is in part supported by grants from the Office of Naval Research and the McDonnell-Pew foundation to P. B. We would like to thank Yosi Rinott for useful discussions.
Funders:
Funding AgencyGrant Number
Office of Naval ResearchUNSPECIFIED
McDonnell-Pew foundationUNSPECIFIED
Record Number:CaltechAUTHORS:BALnc91b
Persistent URL:http://resolver.caltech.edu/CaltechAUTHORS:BALnc91b
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:13791
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:18 Jun 2009 18:29
Last Modified:26 Dec 2012 10:54

Repository Staff Only: item control page