CaltechAUTHORS
  A Caltech Library Service

The VC-Dimension versus the Statistical Capacity of Multilayer Networks

Ji, Chuanyi and Psaltis, Demetri (1992) The VC-Dimension versus the Statistical Capacity of Multilayer Networks. In: Advances in Neural Information Processing Systems 4. Advances in Neural Information Processing Systems. No.4. Morgan Kaufmann , San Mateo, CA, pp. 928-935. ISBN 1-55860-222-4. https://resolver.caltech.edu/CaltechAUTHORS:20160121-163657790

[img] PDF - Published Version
See Usage Policy.

1534Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20160121-163657790

Abstract

A general relationship is developed between the VC-dimension and the statistical lower epsilon-capacity which shows that the VC-dimension can be lower bounded (in order) by the statistical lower epsilon-capacity of a network trained with random samples. This relationship explains quantitatively how generalization takes place after memorization, and relates the concept of generalization (consistency) with the capacity of the optimal classifier over a class of classifiers with the same structure and the capacity of the Bayesian classifier. Furthermore, it provides a general methodology to evaluate a lower bound for the VC-dimension of feedforward multilayer neural networks. This general methodology is applied to two types of networks which are important for hardware implementations: two layer (N - 2L - 1) networks with binary weights, integer thresholds for the hidden units and zero threshold for the output unit, and a single neuron ((N - 1) networks) with binary weigths and a zero threshold. Specifically, we obtain O(W/lnL)≤ d_2 ≤ O(W), and d_1 ~ O(N). Here W is the total number of weights of the (N - 2L - 1) networks. d_1 and d_2 represent the VC-dimensions for the (N - 1) and (N - 2L - 1) networks respectively.


Item Type:Book Section
Related URLs:
URLURL TypeDescription
http://papers.nips.cc/paper/481-the-vc-dimension-versus-the-statistical-capacity-of-multilayer-networksOrganizationArticle
Additional Information:© 1992 Morgan Kaufmann. The authors would like to thank Yaser Abu-Mostafa and David Haussler for helpful discussions. The support of AFOSR and DARPA is gratefully acknowledged
Funders:
Funding AgencyGrant Number
Air Force Office of Scientific Research (AFOSR)UNSPECIFIED
Defense Advanced Research Projects Agency (DARPA)UNSPECIFIED
Series Name:Advances in Neural Information Processing Systems
Issue or Number:4
Record Number:CaltechAUTHORS:20160121-163657790
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20160121-163657790
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:63859
Collection:CaltechAUTHORS
Deposited By: Kristin Buxton
Deposited On:22 Jan 2016 22:24
Last Modified:03 Oct 2019 09:32

Repository Staff Only: item control page