A Study of Aggregation of Long Time-series Input for LSTM Neural Networks

Abstract

Long Short-Term Memory (LSTM) recurrent neural networks are widely utilized for sequential data modeling, but their capacity to retain meaningful historical state diminishes over very long time horizons. Standard industrial practice divides sequences into uniform time windows and computes simple arithmetic means, treating distant and recent observations with equal granularity.

In this work, we present an exponential temporal aggregation method where window lengths scale exponentially into the past. Combined with expressive aggregation functions (median, maximum, and quartile features), this approach preserves sharp recent dynamics while summarizing long-term historical context. Evaluated across seven diverse real-world time-series datasets, the method yields consistent improvements of 6% to 27% in forecasting accuracy.

Read on OpenReview Download PDF Google Scholar

Recommended citation: Nitzan Farhi, Yuval Shavitt, "A Study of Aggregation of Long Time-series Input for LSTM Neural Networks." OpenReview, 2022.
Download Paper