A Caltech Library Service

The VC-Dimension versus the Statistical Capacity of Multilayer Networks

Ji, Chuanyi and Psaltis, Demetri (1992) The VC-Dimension versus the Statistical Capacity of Multilayer Networks. In: Advances in Neural Information Processing Systems 4. Advances in Neural Information Processing Systems. No.4. Morgan Kaufmann , San Mateo, CA, pp. 928-935. ISBN 1-55860-222-4.

[img] PDF - Published Version
See Usage Policy.


Use this Persistent URL to link to this item:


A general relationship is developed between the VC-dimension and the statistical lower epsilon-capacity which shows that the VC-dimension can be lower bounded (in order) by the statistical lower epsilon-capacity of a network trained with random samples. This relationship explains quantitatively how generalization takes place after memorization, and relates the concept of generalization (consistency) with the capacity of the optimal classifier over a class of classifiers with the same structure and the capacity of the Bayesian classifier. Furthermore, it provides a general methodology to evaluate a lower bound for the VC-dimension of feedforward multilayer neural networks. This general methodology is applied to two types of networks which are important for hardware implementations: two layer (N - 2L - 1) networks with binary weights, integer thresholds for the hidden units and zero threshold for the output unit, and a single neuron ((N - 1) networks) with binary weigths and a zero threshold. Specifically, we obtain O(W/lnL)≤ d_2 ≤ O(W), and d_1 ~ O(N). Here W is the total number of weights of the (N - 2L - 1) networks. d_1 and d_2 represent the VC-dimensions for the (N - 1) and (N - 2L - 1) networks respectively.

Item Type:Book Section
Related URLs:
URLURL TypeDescription
Additional Information:© 1992 Morgan Kaufmann. The authors would like to thank Yaser Abu-Mostafa and David Haussler for helpful discussions. The support of AFOSR and DARPA is gratefully acknowledged
Funding AgencyGrant Number
Air Force Office of Scientific Research (AFOSR)UNSPECIFIED
Defense Advanced Research Projects Agency (DARPA)UNSPECIFIED
Series Name:Advances in Neural Information Processing Systems
Issue or Number:4
Record Number:CaltechAUTHORS:20160121-163657790
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:63859
Deposited On:22 Jan 2016 22:24
Last Modified:03 Oct 2019 09:32

Repository Staff Only: item control page