CaltechAUTHORS
  A Caltech Library Service

Long-term Forecasting using Tensor-Train RNNs

Yu, Rose and Zheng, Stephan and Anandkumar, Anima and Yue, Yisong (2017) Long-term Forecasting using Tensor-Train RNNs. . (Submitted) http://resolver.caltech.edu/CaltechAUTHORS:20190205-113450468

[img] PDF - Submitted Version
See Usage Policy.

2103Kb

Use this Persistent URL to link to this item: http://resolver.caltech.edu/CaltechAUTHORS:20190205-113450468

Abstract

We present Tensor-Train RNN (TT-RNN), a novel family of neural sequence architectures for multivariate forecasting in environments with nonlinear dynamics. Long-term forecasting in such systems is highly challenging, since there exist long-term temporal dependencies, higher-order correlations and sensitivity to error propagation. Our proposed tensor recurrent architecture addresses these issues by learning the nonlinear dynamics directly using higher order moments and high-order state transition functions. Furthermore, we decompose the higher-order structure using the tensor-train (TT) decomposition to reduce the number of parameters while preserving the model performance. We theoretically establish the approximation properties of Tensor-Train RNNs for general sequence inputs, and such guarantees are not available for usual RNNs. We also demonstrate significant long-term prediction improvements over general RNN and LSTM architectures on a range of simulated environments with nonlinear dynamics, as well on real-world climate and traffic data.


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
http://arxiv.org/abs/1711.00073arXivDiscussion Paper
Record Number:CaltechAUTHORS:20190205-113450468
Persistent URL:http://resolver.caltech.edu/CaltechAUTHORS:20190205-113450468
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:92672
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:05 Feb 2019 19:39
Last Modified:05 Feb 2019 19:39

Repository Staff Only: item control page