Monotonicity: Theory and Implementation

Abstract

We present a systematic method for incorporating prior knowledge (hints) into the learning-from-examples paradigm. The hints are represented in a canonical form that is compatible with descent techniques for learning. We focus in particular on the monotonicity hint, which states that the function to be learned is monotonic in some or all of the input variables. The application of monotonicity hints is demonstrated on two real-world problems-a credit card application task, and a problem in medical diagnosis. We report experimental results which show that using monotonicity hints leads to a statistically significant improvement in performance on both problems. Monotonicity is also analyzed from a theoretical perspective. We consider the class M of monotonically increasing binary output functions. Necessary and sufficient conditions for monotonic separability of a dichotomy are proven. The capacity of M is shown to depend heavily on the input distribution.