Hassibi, Babak and Stork, David G. and Wolff, Gregory and Watanabe, Takahiro (1994) Optimal Brain Surgeon: Extensions and performance comparison. In: Advances in Neural Information Processing Systems 6. Morgan Kaufmann , San Francisco, pp. 263-270. ISBN 9781558603226. https://resolver.caltech.edu/CaltechAUTHORS:20150219-074935323
![]() |
PDF
- Published Version
See Usage Policy. 1681Kb |
Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20150219-074935323
Abstract
We extend Optimal Brain Surgeon (OBS) - a second-order method for pruning networks - to allow for general error measures, and explore a reduced computational and storage implementation via a dominant eigenspace decomposition. Simulations on nonlinear, noisy pattern classification problems reveal that OBS does lead to improved generalization, and performs favorably in comparison with Optimal Brain Damage (OBD). We find that the required retraining steps in OBD may lead to inferior generalization, a result that can be interpreted as due to injecting noise back into the system. A common technique is to stop training of a large network at the minimum validation error. We found that the test error could be reduced even further by means of OBS (but not OBD) pruning. Our results justify the t → o approximation used in OBS and indicate why retraining in a highly pruned network may lead to inferior performance.
Item Type: | Book Section | ||||||
---|---|---|---|---|---|---|---|
Related URLs: |
| ||||||
Additional Information: | Copyright 1994. Thanks to T. Kailath for support of B.H. through grants AFOSR 91-0060 and DAAL03-91-C-0010. | ||||||
Funders: |
| ||||||
Record Number: | CaltechAUTHORS:20150219-074935323 | ||||||
Persistent URL: | https://resolver.caltech.edu/CaltechAUTHORS:20150219-074935323 | ||||||
Usage Policy: | No commercial reproduction, distribution, display or performance rights in this work are provided. | ||||||
ID Code: | 54982 | ||||||
Collection: | CaltechAUTHORS | ||||||
Deposited By: | Shirley Slattery | ||||||
Deposited On: | 04 Mar 2015 17:21 | ||||||
Last Modified: | 03 Oct 2019 08:02 |
Repository Staff Only: item control page