Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published July 1993 | Published
Journal Article Open

On loss functions which minimize to conditional expected values and posterior probabilities


A loss function, or objective function, is a function used to compare parameters when fitting a model to data. The loss function gives a distance between the model output and the desired output. Two common examples are the squared-error loss function and the cross entropy loss function. Minimizing the mean-square error loss function is equivalent to minimizing the mean square difference between the model output and the expected value of the output given a particular input. This property of minimization to the expected value is formalized as P-admissibility. The necessary and sufficient conditions for P-admissibility, leading to a parametric description of all P-admissible loss functions, are found. In particular, it is shown that two of the simplest members of this class of functions are the squared error and the cross entropy loss functions. One application of this work is in the choice of a loss function for training neural networks to provide probability estimates.

Additional Information

© 1993 IEEE. Manuscript received December 5, 1991. This work was supported in part by DARPA under Grant AFOSR-90-0199 and in part by NSF Grant ENG-8711673. This work was presented in part at the IEEE International Symposium on Information Theory, Budapest, Hungary, June 24-28, 1991. This work carried out in part by the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration.

Attached Files

Published - 00243457.pdf


Files (498.2 kB)
Name Size Download all
498.2 kB Preview Download

Additional details

August 20, 2023
October 20, 2023