Yu, Rose and Zheng, Stephan and Anandkumar, Anima and Yue, Yisong (2017) Long-term Forecasting using Tensor-Train RNNs. . (Submitted) https://resolver.caltech.edu/CaltechAUTHORS:20190205-113450468
![]() |
PDF
- Submitted Version
See Usage Policy. 2MB |
Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20190205-113450468
Abstract
We present Tensor-Train RNN (TT-RNN), a novel family of neural sequence architectures for multivariate forecasting in environments with nonlinear dynamics. Long-term forecasting in such systems is highly challenging, since there exist long-term temporal dependencies, higher-order correlations and sensitivity to error propagation. Our proposed tensor recurrent architecture addresses these issues by learning the nonlinear dynamics directly using higher order moments and high-order state transition functions. Furthermore, we decompose the higher-order structure using the tensor-train (TT) decomposition to reduce the number of parameters while preserving the model performance. We theoretically establish the approximation properties of Tensor-Train RNNs for general sequence inputs, and such guarantees are not available for usual RNNs. We also demonstrate significant long-term prediction improvements over general RNN and LSTM architectures on a range of simulated environments with nonlinear dynamics, as well on real-world climate and traffic data.
Item Type: | Report or Paper (Discussion Paper) | ||||||
---|---|---|---|---|---|---|---|
Related URLs: |
| ||||||
ORCID: |
| ||||||
Record Number: | CaltechAUTHORS:20190205-113450468 | ||||||
Persistent URL: | https://resolver.caltech.edu/CaltechAUTHORS:20190205-113450468 | ||||||
Usage Policy: | No commercial reproduction, distribution, display or performance rights in this work are provided. | ||||||
ID Code: | 92672 | ||||||
Collection: | CaltechAUTHORS | ||||||
Deposited By: | Tony Diaz | ||||||
Deposited On: | 05 Feb 2019 19:39 | ||||||
Last Modified: | 09 Mar 2020 13:19 |
Repository Staff Only: item control page