CaltechAUTHORS
  A Caltech Library Service

Score Function Features for Discriminative Learning: Matrix and Tensor Framework

Janzamin, Majid and Sedghi, Hanie and Anandkumar, Anima (2014) Score Function Features for Discriminative Learning: Matrix and Tensor Framework. . (Unpublished) https://resolver.caltech.edu/CaltechAUTHORS:20190401-162918161

[img] PDF - Submitted Version
See Usage Policy.

352Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20190401-162918161

Abstract

Feature learning forms the cornerstone for tackling challenging learning problems in domains such as speech, computer vision and natural language processing. In this paper, we consider a novel class of matrix and tensor-valued features, which can be pre-trained using unlabeled samples. We present efficient algorithms for extracting discriminative information, given these pre-trained features and labeled samples for any related task. Our class of features are based on higher-order score functions, which capture local variations in the probability density function of the input. We establish a theoretical framework to characterize the nature of discriminative information that can be extracted from score-function features, when used in conjunction with labeled samples. We employ efficient spectral decomposition algorithms (on matrices and tensors) for extracting discriminative components. The advantage of employing tensor-valued features is that we can extract richer discriminative information in the form of an overcomplete representations. Thus, we present a novel framework for employing generative models of the input for discriminative learning.


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
http://arxiv.org/abs/1412.2863arXivDiscussion Paper
Additional Information:M. Janzamin thanks Rina Panigrahy for useful discussions. M. Janzamin is supported by NSF Award CCF-1219234. H. Sedghi is supported by ONR Award N00014-14-1-0665. A. Anandkumar is supported in part by Microsoft Faculty Fellowship, NSF Career award CCF-1254106, NSF Award CCF-1219234, ARO YIP Award W911NF-13-1-0084 and ONR Award N00014-14-1-0665.
Funders:
Funding AgencyGrant Number
NSFCCF-1219234
Office of Naval Research (ONR)N00014-14-1-0665
Microsoft Faculty FellowshipUNSPECIFIED
NSFCCF-1254106
Army Research Office (ARO)W911NF-13-1-0084
Subject Keywords:Feature learning, pre-training, score function, spectral decomposition methods, tensor methods
Record Number:CaltechAUTHORS:20190401-162918161
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20190401-162918161
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:94344
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:02 Apr 2019 22:46
Last Modified:03 Oct 2019 21:03

Repository Staff Only: item control page