CaltechAUTHORS
  A Caltech Library Service

Optimal Brain Surgeon: Extensions and performance comparison.

Hassibi, Babak and Stork, David G. and Wolff, Gregory and Watanabe, Takahiro (1994) Optimal Brain Surgeon: Extensions and performance comparison. In: Advances in Neural Information Processing Systems 6. Morgan Kaufmann , San Francisco, pp. 263-270. ISBN 9781558603226. https://resolver.caltech.edu/CaltechAUTHORS:20150219-074935323

[img] PDF - Published Version
See Usage Policy.

1681Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20150219-074935323

Abstract

We extend Optimal Brain Surgeon (OBS) - a second-order method for pruning networks - to allow for general error measures, and explore a reduced computational and storage implementation via a dominant eigenspace decomposition. Simulations on nonlinear, noisy pattern classification problems reveal that OBS does lead to improved generalization, and performs favorably in comparison with Optimal Brain Damage (OBD). We find that the required retraining steps in OBD may lead to inferior generalization, a result that can be interpreted as due to injecting noise back into the system. A common technique is to stop training of a large network at the minimum validation error. We found that the test error could be reduced even further by means of OBS (but not OBD) pruning. Our results justify the t → o approximation used in OBS and indicate why retraining in a highly pruned network may lead to inferior performance.


Item Type:Book Section
Related URLs:
URLURL TypeDescription
http://papers.nips.cc/paper/749-optimal-brain-surgeon-extensions-and-performance-comparisonsOrganizationNIPS Proceedings. Book chapter
Additional Information:Copyright 1994. Thanks to T. Kailath for support of B.H. through grants AFOSR 91-0060 and DAAL03-91-C-0010.
Funders:
Funding AgencyGrant Number
Air Force Office of Scientific Research (AFOSR)91-0060
Army Research Office (ARO)DAAL03-91-C-0010
Record Number:CaltechAUTHORS:20150219-074935323
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20150219-074935323
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:54982
Collection:CaltechAUTHORS
Deposited By: Shirley Slattery
Deposited On:04 Mar 2015 17:21
Last Modified:03 Oct 2019 08:02

Repository Staff Only: item control page