Forecasting Using Elman Recurrent Neural Network

Deep neural networks have proven to perform optimal forecasts even with the presence of noisyand non-linear nature of time series data. In thispaper, a hybrid deep neural network consisting of Convolutional Neural Networks (CNN) and Long Short Term Memory (LSTM) architecture have been proposed. The model combines the convolutional layer's capability of feature extraction along with the LSTM's feature of learning long term sequential dependencies. The experiments were performed on two datasets and compared with four other approaches: Recurrent Neural Network (RNN), LSTM, Gated Recurrent Unit (GRU) and Bidirectional LSTM. All five models are evaluated and compared with one step ahead forecasting. The proposed hybrid CNN-LSTM outperformed other modelsfor both datasets showing robustness against error.

2016

During recent decades, several studies have been conducted in the field of weather forecasting providing various promising forecasting models. Nevertheless, the accuracy of the predictions still remains a challenge. In this paper a new forecasting approach is proposed: it implements a deep neural network based on a powerful feature extraction. The model is capable of deducing the irregular structure, non-linear trends and significant representations as features learnt from the data. It is a 6-layered deep architecture with 4 hidden units of Restricted Boltzmann Machine (RBM). The extracts from the last hidden layer are pre-processed, to support the accuracy achieved by the forecaster. The forecaster is a 2-layer ANN model with 35 hidden units for predicting the future intervals. It captures the correlations and regression patterns of the current sample related to the previous terms by using the learnt deep-hierarchal representations of data as an input to the forecaster.

Informs Journal on Computing, 1993

We have studied neural networks as models for time series forecasting, and our research compares the Box-Jenkins method against the neural network method for long and short term memory series. Our work was inspired by previously published works that yielded inconsistent results about comparative performance. We have since experimented with 16 time series of di ering complexity using neural networks. The performance of the neural networks is compared with that of the Box-Jenkins method. Our experiments indicate that for time series with long memory, both methods produced comparable results. However, for series with short memory, neural networks outperformed the Box-Jenkins model. Because neural networks can be easily built for multiple-step-ahead forecasting, they present a better long term forecast model than the Box-Jenkins method. We discussed the representation ability, the model building process and the applicability of the neural net approach. Neural networks appear to provide a promising alternative for time series forecasting.

2018

Time series datasets contain a collection of observations in time and this additional temporal component differentiates time series with normal datasets. Studies of forecasting time series datasets are particularly popular as they improve financial planning, sales prediction, average rainfall forecast, internet traffic rates, business trend forecasting, weather forecast, contagious disease spread, etc. Time series forecasting facilitates the decision process in various domains such as business by predicting timely sales information thereby improving management of product manufacturing output. In medical domain, predicting patient health improves medical treatment. The environmental and physical time series forecasting such as rainfall, disasters prediction would improve the contingency plans. On an overall basis time series prediction improves the way humans live. Artificial neural networks are increasingly becoming successful in predicting the time series data through regression-ba...

International Journal of Neural Systems, 2021

In recent years, deep learning techniques have outperformed traditional models in many machine learning tasks. Deep neural networks have successfully been applied to address time series forecasting problems, which is a very important topic in data mining. They have proved to be an effective solution given their capacity to automatically learn the temporal dependencies present in time series. However, selecting the most convenient type of deep neural network and its parametrization is a complex task that requires considerable expertise. Therefore, there is a need for deeper studies on the suitability of all existing architectures for different forecasting tasks. In this work, we face two main challenges: a comprehensive review of the latest works using deep learning for time series forecasting; and an experimental study comparing the performance of the most popular architectures. The comparison involves a thorough analysis of seven types of deep learning models in terms of accuracy and efficiency. We evaluate the rankings and distribution of results obtained with the proposed models under many different architecture configurations and training hyperparameters. The datasets used comprise more than 50000 time series divided into 12 different forecasting problems. By training more than 38000 models on these data, we provide the most extensive deep learning study for time series forecasting. Among all studied models, the results show that long short-term memory (LSTM) and convolutional networks (CNN) are the best alternatives, with LSTMs obtaining the most accurate forecasts. CNNs achieve comparable performance with less variability of results under different parameter configurations, while also being more efficient.

Recent studies have shown the classification and prediction power of the Neural Networks. It has been demonstrated that a NN can approximate any continuous function. Neural networks have been successfully used for forecasting of financial data series. The classical methods used for time series prediction like Box-Jenkins or ARIMA assumes that there is a linear relationship between inputs and outputs. Neural Networks have the advantage that can approximate nonlinear functions. In this paper we compared the performances of different feed forward and recurrent neural networks and training algorithms for predicting the exchange rate EUR/RON and USD/RON. We used data series with daily exchange rates starting from 2005 until 2013.

A Survey on Deep Learning for Time-Series Forecasting, 2021

Deep learning, one of the most remarkable techniques of machine learning, has been a major success in many fields, including image processing, speech recognition, and text understanding. It is powerful engines capable of learning arbitrary mapping functions, not require a scaled or stationary time series as input, support multivariate inputs, and support multi-step outputs. All of these features together make deep learning useful tools when dealing with more complex time series prediction problems involving large amounts of data, and multiple variables with complex relationships. This paper provides an overview of the most common Deep Learning types for time series forecasting, Explain the relationships between deep learning models and classical approaches to time series forecasting. A brief background of the particular challenges presents in time-series data and the most common deep learning techniques that are often used for time series forecasting is provided. Previous studies that applied deep learning to time series are reviewed.

Proceedings of the International Joint Conference on Neural Networks, 2012

This paper provides a comparison between two methods for time series forecasting. The first method is based on traditional recurrent neural networks (RNNs) while the second method is based in Reservoir Computing (RC). Reservoir Computing is a new paradigm that offers an intuitive methodology for using the temporal processing power of RNNs without the inconvenience of training them. So we decided to compare the advantages / disadvantages of using Reservoir Computing and RNNs in the problem of time series forecasting. The first method uses a Nonlinear Autoregressive Network with exogenous inputs (NARX). Optimization was carried out on the NARX architecture through an optimization procedure focused on the best MSE metrics in the training set. The second method, called RCDESIGN, combines an evolutionary algorithm with Reservoir Computing and simultaneously looks for the best values of parameters, topology and weight matrices without rescaling the reservoir by the spectral radius. Nevertheless RCDESIGN has yielded fast tracking and excellent performance in some benchmark problems including the Narma and Mackey-Glass time-series.

Journal of Physics: Conference Series, 2021

Long Short-Term Memory (LSTM) is one of the developments from Recurrent Neural Network (RNN) architecture. In this paper, we use LSTM architecture for modeling and forecasting the Indonesian Composite Stock Price Index (IHSG) closing value data. We also compare the performance of the LSTM method with the ARIMA and the Radial Basis Function (RBF) Neural Network method. In the implementation, we use both R and Python open source software. For empirical study we use the data from January until August 2020 to see the performance of the considered methods during Covid-19 pandemic periods of time. From the analysis, we found that LSTM performs better than ARIMA, but outperformed by RBF for this data.

2007

In this paper we present an efficient approach for the prediction of two sunspot-related time series, namely the Yearly Sunspot Number and the IR5 Index, that are commonly used for monitoring solar activity. The method is based on exploiting partially recurrent Elman networks and it can be divided into three main steps: the first one consists in a "de-rectification" of the time series under study in order to obtain a new time series whose appearance, similar to a sum of sinusoids, can be modelled by our neural networks much better than the original dataset. After that, we normalize the derectified data so that they have zero mean and unity standard deviation and, finally, train an Elman network with only one input, a recurrent hidden layer and one output using a back-propagation algorithm with variable learning rate and momentum. The achieved results have shown the efficiency of this approach that, although very simple, can perform better than most of the existing solar ac...