CaltechAUTHORS
  A Caltech Library Service

Validation of Average Error Rate Over Classifiers

Bax, Eric (1997) Validation of Average Error Rate Over Classifiers. California Institute of Technology . (Unpublished) http://resolver.caltech.edu/CaltechCSTR:1997.cs-tr-97-17

[img]
Preview
Postscript
See Usage Policy.

293Kb
[img]
Preview
Other (Adobe PDF (150KB))
See Usage Policy.

146Kb

Use this Persistent URL to link to this item: http://resolver.caltech.edu/CaltechCSTR:1997.cs-tr-97-17

Abstract

We examine methods to estimate the average and variance of test error rates over a set of classifiers. We begin with the process of drawing a classifier at random for each example. Given validation data, the average test error rate can be estimated as if validating a single classifier. Given the test example inputs, the variance can be computed exactly. Next, we consider the process of drawing a classifier at random and using it on all examples. Once again, the expected test error rate can be validated as if validating a single classifier. However, the variance must be estimated by validating all classifers, which yields loose or uncertain bounds.


Item Type:Report or Paper (Technical Report)
Group:Computer Science Technical Reports
Record Number:CaltechCSTR:1997.cs-tr-97-17
Persistent URL:http://resolver.caltech.edu/CaltechCSTR:1997.cs-tr-97-17
Usage Policy:You are granted permission for individual, educational, research and non-commercial reproduction, distribution, display and performance of this work in any format.
ID Code:26825
Collection:CaltechCSTR
Deposited By: Imported from CaltechCSTR
Deposited On:30 Apr 2001
Last Modified:26 Dec 2012 14:06

Repository Staff Only: item control page