Baldi, Pierre and Chauvin, Yves (1991) Temporal evolution of generalization during learning in linear networks. Neural Computation, 3 (4). pp. 589-603. ISSN 0899-7667 http://resolver.caltech.edu/CaltechAUTHORS:BALnc91b
- Published Version
See Usage Policy.
Use this Persistent URL to link to this item: http://resolver.caltech.edu/CaltechAUTHORS:BALnc91b
We study generalization in a simple framework of feedforward linear networks with n inputs and n outputs, trained from examples by gradient descent on the usual quadratic error function. We derive analytical results on the behavior of the validation function corresponding to the LMS error function calculated on a set of validation patterns. We show that the behavior of the validation function depends critically on the initial conditions and on the characteristics of the noise. Under certain simple assumptions, if the initial weights are sufficiently small, the validation function has a unique minimum corresponding to an optimal stopping time for training for which simple bounds can be calculated. There exists also situations where the validation function can have more complicated and somewhat unexpected behavior such as multiple local minima (at most n) of variable depth and long but finite plateau effects. Additional results and possible extensions are briefly discussed.
|Additional Information:||© 1991 Massachusetts Institute of Technology. Received 1 February 1991; accepted 13 April 1991. Posted Online March 13, 2008. This work is in part supported by grants from the Office of Naval Research and the McDonnell-Pew foundation to P. B. We would like to thank Yosi Rinott for useful discussions.|
|Usage Policy:||No commercial reproduction, distribution, display or performance rights in this work are provided.|
|Deposited By:||Tony Diaz|
|Deposited On:||18 Jun 2009 18:29|
|Last Modified:||26 Dec 2012 10:54|
Repository Staff Only: item control page