A Caltech Library Service

Invariance Hints and the VC Dimension

Fyfe, William John Andrew (1992) Invariance Hints and the VC Dimension. Computer Science Technical Reports, California Institute of Technology , Pasadena, CA. (Unpublished)

Postscript - Submitted Version
See Usage Policy.

PDF - Submitted Version
See Usage Policy.


Use this Persistent URL to link to this item:


We are interested in having a neural network learn an unknown function f. If the function satisfies an invariant of some sort, such as f is an odd function, then we want to be able to take advantage of this information and not have the network deduce the invariant based on an example of f. The invariant might be defined in terms of an explicit transformation of the input space under which f is constant. In this case it is possible to build a network thatnecessarily satisfies the invariant. In general, we define the invariant in terms of a partition of the input space such that if x, x' are in the same partition element then f (x) = f (x'). An example of the invariant would be a pair (x, x') taken from a single partition element. We can combine examples of the invariant with examples of the function in the learning process. The goal is to substitute examples of the invariant for examples of the function; the extent to which we can actually do this depends on the appropriate VC dimensions. Simulations verify, at least in simple cases, that examples of the invariant do aid the learning process.

Item Type:Report or Paper (Technical Report)
Additional Information:© William John Andrew Fyfe. 1992 California Institute of Technology. Submitted 26 May, 1992. Many people have had a band in this thesis, in many different ways. The content was influenced enotmously by my advisor, Dr. Yaser Abu-Mostafa, who, over the years, assisted me along those research paths that led to this thesis, as well as those that did not. The thesis was reviewed by Doctors Alan Barr, Carver Mead, Edward Posner and Richard Wilson, and I am grateful for their comments and suggestions. My research group, over the years, provided extremely useful feedback: Amir Atiya, Ruth Erlanson, Allen Knutson, Jack Lutz, David Schweizer. Funding for my graduate studies came from a number of sources: the Natural Sciences and Engineering Research Council of Canada, the Air Force Office of Scientific Research, Hughes, and of course Caltech itself, which provided not only financial support, but a place to work. Essential to the thesis was the well-being of its author, and essential to that were the many people around me. Some, John, David, Steve and Steve, have left to move on to other things; others, Pat, Barry and Rob, remain, and have struggled with me. Finally, graduating with a thesis implies having started here some time ago. Getting here depended on many people who taught and inspired me along the way. Among the many, the names Ostlund, Ponzo, and Wilson stick out. And, of course, above all, my parents.
Group:Computer Science Technical Reports
Funding AgencyGrant Number
Natural Sciences and Engineering Research Council of Canada (NSERC)UNSPECIFIED
Air Force Office of Scientific Research (AFOSR)UNSPECIFIED
Series Name:Computer Science Technical Reports
Record Number:CaltechCSTR:1992.cs-tr-92-20
Persistent URL:
Usage Policy:You are granted permission for individual, educational, research and non-commercial reproduction, distribution, display and performance of this work in any format.
ID Code:26741
Deposited By: Imported from CaltechCSTR
Deposited On:25 Apr 2001
Last Modified:03 Oct 2019 03:17

Repository Staff Only: item control page