CaltechAUTHORS
  A Caltech Library Service

Kernel Mode Decomposition and programmable/interpretable regression networks

Owhadi, Houman and Scovel, Clint and Yoo, Gene Ryan (2019) Kernel Mode Decomposition and programmable/interpretable regression networks. . (Unpublished) https://resolver.caltech.edu/CaltechAUTHORS:20190923-153747161

[img] PDF - Submitted Version
See Usage Policy.

5Mb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20190923-153747161

Abstract

Mode decomposition is a prototypical pattern recognition problem that can be addressed from the (a priori distinct) perspectives of numerical approximation, statistical inference and deep learning. Could its analysis through these combined perspectives be used as a Rosetta stone for deciphering mechanisms at play in deep learning? Motivated by this question we introduce programmable and interpretable regression networks for pattern recognition and address mode decomposition as a prototypical problem. The programming of these networks is achieved by assembling elementary modules decomposing and recomposing kernels and data. These elementary steps are repeated across levels of abstraction and interpreted from the equivalent perspectives of optimal recovery, game theory and Gaussian process regression (GPR). The prototypical mode/kernel decomposition module produces an optimal approximation (w_1,w_2,⋯,w_m) of an element (v_1,v_2,…,v_m) of a product of Hilbert subspaces of a common Hilbert space from the observation of the sum v:=v_1+⋯+v_m. The prototypical mode/kernel recomposition module performs partial sums of the recovered modes w_i based on the alignment between each recovered mode w_i and the data v. We illustrate the proposed framework by programming regression networks approximating the modes v_i = a_i(t)y_i(θ_i(t)) of a (possibly noisy) signal ∑_iv_i when the amplitudes a_i, instantaneous phases θ_i and periodic waveforms y_i may all be unknown and show near machine precision recovery under regularity and separation assumptions on the instantaneous amplitudes a_i and frequencies θ_i. The structure of some of these networks share intriguing similarities with convolutional neural networks while being interpretable, programmable and amenable to theoretical analysis.


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
http://arxiv.org/abs/1907.08592arXivDiscussion Paper
ORCID:
AuthorORCID
Owhadi, Houman0000-0002-5677-1600
Scovel, Clint0000-0001-7757-3411
Additional Information:The authors gratefully acknowledge support by the Air Force Office of Scientific Research under award number FA9550-18-1-0271 (Games for Computation and Learning).
Funders:
Funding AgencyGrant Number
Air Force Office of Scientific Research (AFOSR)FA9550-18-1-0271
Record Number:CaltechAUTHORS:20190923-153747161
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20190923-153747161
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:98806
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:23 Sep 2019 22:46
Last Modified:03 Oct 2019 21:44

Repository Staff Only: item control page