A Caltech Library Service

Robust Deep Networks with Randomized Tensor Regression Layers

Kolbeinsson, Arinbjörn and Kossaifi, Jean and Panagakis, Yannis and Bulat, Adrian and Anandkumar, Anima and Tzoulaki, Ioanna and Matthews, Paul (2019) Robust Deep Networks with Randomized Tensor Regression Layers. . (Unpublished)

[img] PDF - Submitted Version
See Usage Policy.


Use this Persistent URL to link to this item:


In this paper, we propose a novel randomized tensor decomposition for tensor regression. It allows to stochastically approximate the weights of tensor regression layers by randomly sampling in the low-rank subspace. We theoretically and empirically establish the link between our proposed stochastic rank-regularization and the dropout on low-rank tensor regression. This acts as an additional stochastic regularization on the regression weight, which, combined with the deterministic regularization imposed by the low-rank constraint, improves both the performance and robustness of neural networks augmented with it. In particular, it makes the model more robust to adversarial attacks and random noise, without requiring any adversarial training. We perform a thorough study of our method on synthetic data, object classification on the CIFAR100 and ImageNet datasets, and large scale brain-age prediction on UK Biobank brain MRI dataset. We demonstrate superior performance in all cases, as well as significant improvement in robustness to adversarial attacks and random noise.

Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription Paper
Additional Information:This research has been conducted using the UK Biobank Resource under Application Number 18545.
Funding AgencyGrant Number
UK Biobank18545
Record Number:CaltechAUTHORS:20190905-154237568
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:98450
Deposited By: George Porter
Deposited On:06 Sep 2019 14:59
Last Modified:03 Oct 2019 21:41

Repository Staff Only: item control page