Ji, Chuanyi and Psaltis, Demetri (1992) The VC-Dimension versus the Statistical Capacity of Multilayer Networks. In: Advances in Neural Information Processing Systems 4. Advances in Neural Information Processing Systems. No.4. Morgan Kaufmann , San Mateo, CA, pp. 928-935. ISBN 1-55860-222-4. https://resolver.caltech.edu/CaltechAUTHORS:20160121-163657790
![]() |
PDF
- Published Version
See Usage Policy. 1MB |
Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20160121-163657790
Abstract
A general relationship is developed between the VC-dimension and the statistical lower epsilon-capacity which shows that the VC-dimension can be lower bounded (in order) by the statistical lower epsilon-capacity of a network trained with random samples. This relationship explains quantitatively how generalization takes place after memorization, and relates the concept of generalization (consistency) with the capacity of the optimal classifier over a class of classifiers with the same structure and the capacity of the Bayesian classifier. Furthermore, it provides a general methodology to evaluate a lower bound for the VC-dimension of feedforward multilayer neural networks. This general methodology is applied to two types of networks which are important for hardware implementations: two layer (N - 2L - 1) networks with binary weights, integer thresholds for the hidden units and zero threshold for the output unit, and a single neuron ((N - 1) networks) with binary weigths and a zero threshold. Specifically, we obtain O(W/lnL)≤ d_2 ≤ O(W), and d_1 ~ O(N). Here W is the total number of weights of the (N - 2L - 1) networks. d_1 and d_2 represent the VC-dimensions for the (N - 1) and (N - 2L - 1) networks respectively.
Item Type: | Book Section | ||||||
---|---|---|---|---|---|---|---|
Related URLs: |
| ||||||
Additional Information: | © 1992 Morgan Kaufmann. The authors would like to thank Yaser Abu-Mostafa and David Haussler for helpful discussions. The support of AFOSR and DARPA is gratefully acknowledged | ||||||
Funders: |
| ||||||
Series Name: | Advances in Neural Information Processing Systems | ||||||
Issue or Number: | 4 | ||||||
Record Number: | CaltechAUTHORS:20160121-163657790 | ||||||
Persistent URL: | https://resolver.caltech.edu/CaltechAUTHORS:20160121-163657790 | ||||||
Usage Policy: | No commercial reproduction, distribution, display or performance rights in this work are provided. | ||||||
ID Code: | 63859 | ||||||
Collection: | CaltechAUTHORS | ||||||
Deposited By: | INVALID USER | ||||||
Deposited On: | 22 Jan 2016 22:24 | ||||||
Last Modified: | 03 Oct 2019 09:32 |
Repository Staff Only: item control page