Higher order neural network

Purpose: The purpose of this research was to provide a model for predicting time series of financial information based on the Lyapunov representation of information using chaos theory. Method: This research is applied in its purpose, which is conducted using a quantitative approach. The research ranks as descriptive-causal accounting research based on actual information in companies' financial statements. The research method is the "post-event" type and was carried out using chaos theory and Saida's method based on the Lyapunov view. Findings: The findings showed that during the ADF test, the null hypothesis was rejected at a level of less than 5% type 1 error and 95% confidence, and it shows that the data is not static. During the substitution analysis test and its significance level, the behavior of the time series of the main financial ‫ــــــــــــــــــــــــــ‬ ‫ــــ‬ ‫ـــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــ‬

Higher Order Neural Networks (HONNs) have emerged as an important tool for time series prediction and have been successfully applied in many engineering and scientific problems. One of the models in HONNs is a Functional Link Neural Network (FLNN) known to be conveniently used for function approximation and can be extended for pattern recognition with faster convergence rate and lesser computational load compared to ordinary feedforward network like the Multilayer Perceptron (MLP). In training the FLNN, the mostly used algorithm is the Backpropagation (BP) learning algorithm. However, one of the crucial problems with BP learning algorithm is that it can be easily gets trapped on local minima. This paper proposed an alternative learning scheme for the FLNN to be applied on temperature forecasting by using Artificial Bee Colony (ABC) optimization algorithm. The ABC adopted in this work is known to have good exploration and exploitation capabilities in searching optimal weight especially in numerical optimization problems. The result of the prediction made by FLNN-ABC is compared with the original FLNN architecture and toward the end we found that FLNN-ABC gives better result in predicting the next-day ahead prediction.

Functional Link Neural Network (FLNN) has becoming as an important tool for solving non-linear classification problem. This is due to its modest architecture which required less tunable weights for learning as compared to the standard multilayer feed forward network. The most common learning scheme for tuning the weight in FLNN is a Backpropagation (BPlearning) algorithm. However, the learning method by BP-learning algorithm tends to easily get trapped in local minima which affect the performance of FLNN. This paper discussed the implementation of modified Artificial Bee Colony (mABC) as a learning scheme for training the FLNN network in overcoming the drawback of BP-learning scheme. The aim is to introduce an alternative learning scheme that can provide a better solution for training the FLNN network.

An artificial neural network (ANN) based on particle swarm optimization (PSO) was developed for the time series prediction. This hybrid ANN+PSO algorithm was applied on Mackey-Glass series in the short-term prediction x(t + 6) and the long-term prediction x(t + 84), from the current value x(t) and the past values: x(t − 6), x(t − 12), x(t − 18). Four cases were studied, alternating the timedelay parameter as 17 or 30. Also, the first four largest Lyapunov exponents were obtained for different time-delay. Simulation shows that this ANN+PSO method is a very powerful tool for making prediction of chaotic time series.

In this paper, a new self-organizing fuzzy neural network model is presented which is able to learn and reproduce different sequences accurately. Sequence learning is important in performing skillful tasks, such as writing and playing piano. The structure of the proposed network is composed of two parts: 1-sequence identifier which computes a novel sequence identity value based on initial samples of a sequence, and detects the sequence identity based on proper fuzzy rules, and 2-sequence locator, which locates the input sample in the sequence. Therefore, by integrating outputs of these two parts in fuzzy rules, the network is able to produce the proper output based on current state of the sequence. To learn the proposed structure, a gradual learning procedure is proposed. First, learning is performed by adding new fuzzy rules, based on coverage measure, using available correct data. Next, the initialized parameters are fine-tuned, by gradient descent algorithm, based on fed back app...

Nowadays, time series analysis is an important challenge in engineering problems. This paper proposes a Comprehensive Learning Polynomial Autoregressive Model for predicting linear and nonlinear time series. The model is based on the autoregressive model but developed in a polynomial aspect to make it more robust and accurate. The model predicts future values by learning the weights of the weighted sum of the polynomial combination of previous data. The learning process for the hyperparameters and properties of the model in the training phase is performed by the metaheuristic optimization method. Using this model, we can predict nonlinear time series, as well as linear time series. The proposed method was implemented on eight standard stationary and non-stationary large-scale real-world datasets. It outperformed the state-of-the-art methods that use deep learning in seven time series and has better results compared to all other methods in six datasets. Experimental results show the advantage of the model's accuracy over other compared methods on various prediction tasks based on the root-mean-square error.

Time series forecasting has gained much attention due to its many practical applications. Higher-order neural network with recurrent feedback is a powerful technique that has been used successfully for time series forecasting. It maintains fast learning and the ability to learn the dynamics of the time series over time. Network output feedback is the most common recurrent feedback for many recurrent neural network models. However, not much attention has been paid to the use of network error feedback instead of network output feedback. In this study, we propose a novel model, called Ridge Polynomial Neural Network with Error Feedback (RPNN-EF) that incorporates higher order terms, recurrence and error feedback. To evaluate the performance of RPNN-EF, we used four univariate time series with different forecasting horizons, namely star brightness, monthly smoothed sunspot numbers, daily Euro/Dollar exchange rate, and Mackey-Glass time-delay differential equation. We compared the forecasting performance of RPNN-EF with the ordinary Ridge Polynomial Neural Network (RPNN) and the Dynamic Ridge Polynomial Neural Network (DRPNN). Simulation results showed an average 23.34% improvement in Root Mean Square Error (RMSE) with respect to RPNN and an average 10.74% improvement with respect to DRPNN. That means that using network errors during training helps enhance the overall forecasting performance for the network.

The breaking down of a particular problem through problem decomposition has enabled complex problems to be solved efficiently. The two major problem decomposition methods used in cooperative coevolution are synapse and neuron level. The combination of both the problem decomposition as a hybrid problem decomposition has been seen applied in time series prediction. The different problem decomposition methods applied at particular area of a network can share its strengths to solve the problem better, which forms the major motivation. In this paper, we are proposing a combination utilization of two hybrid problem decomposition method for Elman recurrent neural networks and applied to time series prediction. The results reveal that the proposed method has got better results in some datasets when compared to its standalone methods. The results are better in selected cases for proposed method when compared to several other approaches from the literature.

Time series forecasting gets much attention due to its impact on many practical applications. Higher-order neural network with recurrent feedback is a powerful technique which used successfully for forecasting. It maintains fast learning and the ability to learn the dynamics of the series over time. For that, in this paper, we propose a novel model which is called Ridge Polynomial Neural Network with Error-Output Feedbacks (RPNN-EOFs) that combines the properties of higher order and error-output feedbacks. The wellknown MackeyGlass time series is used to test the forecasting capability of RPNN-EOFS. Simulation results showed that the proposed RPNN-EOFs provides better understanding for the MackeyGlass time series with root mean square error equal to 0.00416. This re-sult is smaller than other models in the literature. Therefore, we can conclude that the RPNN-EOFs can be applied successfully for time series forecasting.

Time series forecasting gets much attention due to its impact on many practical applications. Higher-order neural network with recurrent feedback is a powerful technique which used successfully for forecasting. It maintains fast learning and the ability to learn the dynamics of the series over time. For that, in this paper, we propose a novel model which is called Ridge Polynomial Neural Network with Error-Output Feedbacks (RPNN-EOFs) that combines the properties of higher order and error-output feedbacks. The wellknown MackeyGlass time series is used to test the forecasting capability of RPNN-EOFS. Simulation results showed that the proposed RPNN-EOFs provides better understanding for the MackeyGlass time series with root mean square error equal to 0.00416. This result is smaller than other models in the literature. Therefore, we can conclude that the RPNN-EOFs can be applied successfully for time series forecasting. Keywords Time Series forecasting • Ridge polynomial neural network with error-output feedbacks • Higher order neural networks • Recurrent neural networks • Mackey-Glass equation This is a pre-peer-review version of an article published in International Conference on Soft Computing in Data Science.

This study presents a novel application and comparison of higher order neural networks (HONNs) to forecast benchmark chaotic time series. Two models of HONNs were implemented, namely functional link neural network (FLNN) and pi-sigma neural network (PSNN). These models were tested on two benchmark time series; the monthly smoothed sunspot numbers and the Mackey-Glass time-delay differential equation time series. The forecasting performance of the HONNs is compared against the performance of different models previously used in the literature such as fuzzy and neural networks models. Simulation results showed that FLNN and PSNN offer good performance compared to many previously used hybrid models.

Time series forecasting has gained much attention due to its many practical applications. Higher-order neural network with recurrent feedback is a powerful technique that has been used successfully for time series forecasting. It maintains fast learning and the ability to learn the dynamics of the time series over time. Network output feedback is the most common recurrent feedback for many recurrent neural network models. However, not much attention has been paid to the use of network error feedback instead of network output feedback. In this study, we propose a novel model, called Ridge Polynomial Neural Network with Error Feedback (RPNN-EF) that incorporates higher order terms, recurrence and error feedback. To evaluate the performance of RPNN-EF, we used four univariate time series with different forecasting horizons, namely star brightness, monthly smoothed sunspot numbers, daily Euro/Dollar exchange rate, and Mackey-Glass time-delay differential equation. We compared the forecasting performance of RPNN-EF with the ordinary Ridge Polynomial Neural Network (RPNN) and the Dynamic Ridge Polynomial Neural Network (DRPNN). Simulation results showed an average 23.34% improvement in Root Mean Square Error (RMSE) with respect to RPNN and an average 10.74% improvement with respect to DRPNN. That means that using network errors during training helps enhance the overall forecasting performance for the network.

This study presents a novel application and comparison of higher order neural networks (HONNs) to forecast benchmark chaotic time series. Two models of HONNs were implemented, namely functional link neural network (FLNN) and pi-sigma neural network (PSNN). These models were tested on two benchmark time series; the monthly smoothed sunspot numbers and the Mackey-Glass time-delay differential equation time series. The forecasting performance of the HONNs is compared against the performance of different models previously used in the literature such as fuzzy and neural networks models. Simulation results showed that FLNN and PSNN offer good performance compared to many previously used hybrid models.

Time series forecasting has gained much attention due to its many practical applications. Higher-order neural network with recurrent feedback is a powerful technique that has been used successfully for time series forecasting. It maintains fast learning and the ability to learn the dynamics of the time series over time. Network output feedback is the most common recurrent feedback for many recurrent neural network models. However, not much attention has been paid to the use of network error feedback instead of network output feedback. In this study, we propose a novel model, called Ridge Polynomial Neural Network with Error Feedback (RPNN-EF) that incorporates higher order terms, recurrence and error feedback. To evaluate the performance of RPNN-EF, we used four univariate time series with different forecasting horizons, namely star brightness, monthly smoothed sunspot numbers, daily Euro/Dollar exchange rate, and Mackey-Glass time-delay differential equation. We compared the forecasting performance of RPNN-EF with the ordinary Ridge Polynomial Neural Network (RPNN) and the Dynamic Ridge Polynomial Neural Network (DRPNN). Simulation results showed an average 23.34% improvement in Root Mean Square Error (RMSE) with respect to RPNN and an average 10.74% improvement with respect to DRPNN. That means that using network errors during training helps enhance the overall forecasting performance for the network.

The breaking down of a particular problem through problem decomposition has enabled complex problems to be solved efficiently. The two major problem decomposition methods used in cooperative coevolution are synapse and neuron level. The combination of both the problem decomposition as a hybrid problem decomposition has been seen applied in time series prediction. The different problem decomposition methods applied at particular area of a network can share its strengths to solve the problem better, which forms the major motivation. In this paper, we are proposing a combination utilization of two hybrid problem decomposition method for Elman recurrent neural networks and applied to time series prediction. The results reveal that the proposed method has got better results in some datasets when compared to its standalone methods. The results are better in selected cases for proposed method when compared to several other approaches from the literature.

In this paper, a new self-organizing fuzzy neural network model is presented which is able to learn and reproduce different sequences accurately. Sequence learning is important in performing skillful tasks, such as writing and playing piano. The structure of the proposed network is composed of two parts: 1-sequence identifier which computes a novel sequence identity value based on initial samples of a sequence, and detects the sequence identity based on proper fuzzy rules, and 2-sequence locator, which locates the input sample in the sequence. Therefore, by integrating outputs of these two parts in fuzzy rules, the network is able to produce the proper output based on current state of the sequence. To learn the proposed structure, a gradual learning procedure is proposed. First, learning is performed by adding new fuzzy rules, based on coverage measure, using available correct data. Next, the initialized parameters are fine-tuned, by gradient descent algorithm, based on fed back app...

Currency exchange is the trading of one currency against another. FOREX rates are influenced by many correlated economic, political and psychological factors and hence predicting it is an uphill task. Some methods to predict the FOREX rate include statistical analysis, time series analysis, fuzzy systems, neural networks, and hybrid systems. These methods suffer from the problem of accurately predicting the exchange. A Artificial Neural Network (ANN) and a hybrid Neuro-Fuzzy system (ANFIS) are proposed to predict the future rate of the FOREX market. The MLP is used to predict the rise or fall in the exchange rate while the ANFIS model is used to predict the exchange rate for the next day. For the experiment, USDINR exchange rate from the forex market is used. Mean Square Error (MSE) and Mean Absolute Error (MAE) are used as performance indicators. The ANN achieved an MSE of 0.033 and MAE of 0.0002 during training while the ANFIS model achieved an MSE of 0.024 and a MAE of 6.7x10-8. ...

In this paper, we develop an indirect adaptive control structure based on recurrent neural networks. An adaptive emulator inspired from the Real Time recurrent Learning algorithm is presented. Neural network does not learn the plant dynamics but emulates the input-output mapping with a small time window. Thereafter, a controller with a structure similar to neural emulator is described. Both emulator and controller adapt their parameters using an online adaptation algorithm in order to track the process variations. Independent adaptation of networks parameters improve controller performances. Regulation and tracking problems are investigated according to nonlinear system simulations. The satisfactory obtained results show a very good performances in terms of neural emulation and control of nonlinear systems. The contributions of this paper are the validation of our emulator with experimental data from the batch reactor of National Engineering of Gabes, Tunisia and the application of ...

In order to train artificial neural networks, we used a new stochastic optimization algorithm that simulate the plant growing process. It designs two artificial photosynthesis operator and phototropism operator to mimic photosynthesis and phototropism mechanisms, we call it briefly with APPM algorithm. In this algorithm, each individual is called a branch, and the sampled points are regarded as the branch growing trajectory. Phototropism operator is designed to introduce the fitness function value, as well as phototropism operator is used to decide the growing direction. In this paper, we apply APPM algorithm to train the connection weights for artificial neural network (ANN). To assess the performance of our APPM-trained ANN (APPMANN), two real-world problems which named Cleveland heart disease classification problem and sunspot number forecasting problem are adopted. Simulation results show that APPMANN increases the performance significantly when compare with other sophisticated machine learning techniques proposed in recent years.

This study proposes a novel neural-network-based fuzzy group forecasting model for foreign exchange rates prediction. In the proposed model, some single neural network models are first used as predictors for foreign exchange rates prediction. Then these single prediction results produced by each single neural network models are fuzzified into some fuzzy prediction representations. Subsequently, these fuzzified prediction representations are aggregated into a fuzzy group consensus, i.e., aggregated fuzzy prediction representation. Finally, the aggregated prediction representation is defuzzified into a crisp value as the final prediction results. For illustration and testing purposes, a typical numerical example and three typical foreign exchange rates prediction experiments are presented. Experimental results reveal that the proposed model can significantly improve the prediction performance for foreign exchange rates.

The breaking down of a particular problem through problem decomposition has enabled complex problems to be solved efficiently. The two major problem decomposition methods used in cooperative coevolution are synapse and neuron level. The combination of both the problem decomposition as a hybrid problem decomposition has been seen applied in time series prediction. The different problem decomposition methods applied at particular area of a network can share its strengths to solve the problem better, which forms the major motivation. In this paper, we are proposing a combination utilization of two hybrid problem decomposition method for Elman recurrent neural networks and applied to time series prediction. The results reveal that the proposed method has got better results in some datasets when compared to its standalone methods. The results are better in selected cases for proposed method when compared to several other approaches from the literature. Index Terms—Cooperative coevolution, problem decomposition, recurrent network

This paper presents an optimal higher order to forecast temperature event in Batu Pahat, Malaysia by using a Jordan Pi-Sigma Neural Network (JPSN). There are many conventional techniques in dealing with forecasting meteorological issue; however, there are some shortcoming noticed in terms of accuracy and tractability. To solve these problems, we consider evaluating the effects of higher order terms in JPSN for temperature forecasting. The data of temperature measurement in Batu Pahat has been used in order to validate the network model by utilising the backpropagation training algorithm. The results of the prediction made by JPSN were compared with the widely known Multilayer Perceptron. Towards the end, we found that the JPSN of Order 2 gives the best results in predicting the next-day ahead prediction, thus can be used for temperature forecasting with acceptable lower prediction error.

The ability to model the behaviour of arbitrary dynamic system is one of the most useful properties of recurrent networks. Dynamic ridge polynomial neural network (DRPNN) is a recurrent neural network used for time series forecasting. Despite the potential and capability of the DRPNN, stability problems could occur in the DRPNN due to the existence of the recurrent feedback. Therefore, in this study, a sufficient condition based on an approach that uses adaptive learning rate is developed by introducing a Lyapunov function. To compare the performance of the proposed solution with the existing solution, which is derived based on the stability theorem for a feedback network, we used six time series, namely Darwin sea level pressure, monthly smoothed sunspot numbers, Lorenz, Santa Fe laser, daily Euro/Dollar exchange rate and Mackey-Glass time-delay differential equation. Simulation results proved the stability of the proposed solution and showed an average 21.45% improvement in Root Mean Square Error (RMSE) with respect to the existing solution. Furthermore, the proposed solution is faster than the existing solution. This is due to the fact that the proposed solution solves network size restriction found in the existing solution and takes advantage of the calculated dynamic system variable to check the stability, unlike the existing solution that needs more calculation steps.

Time series forecasting has gained much attention due to its many practical applications. Higher-order neural network with recurrent feedback is a powerful technique that has been used successfully for time series forecasting. It maintains fast learning and the ability to learn the dynamics of the time series over time. Network output feedback is the most common recurrent feedback for many recurrent neural network models. However, not much attention has been paid to the use of network error feedback instead of network output feedback. In this study, we propose a novel model, called Ridge Polynomial Neural Network with Error Feedback (RPNN-EF) that incorporates higher order terms, recurrence and error feedback. To evaluate the performance of RPNN-EF, we used four univariate time series with different forecasting horizons, namely star brightness, monthly smoothed sunspot numbers, daily Euro/Dollar exchange rate, and Mackey-Glass time-delay differential equation. We compared the forecasting performance of RPNN-EF with the ordinary Ridge Polynomial Neural Network (RPNN) and the Dynamic Ridge Polynomial Neural Network (DRPNN). Simulation results showed an average 23.34% improvement in Root Mean Square Error (RMSE) with respect to RPNN and an average 10.74% improvement with respect to DRPNN. That means that using network errors during training helps enhance the overall forecasting performance for the network.

This study presents a novel application and comparison of higher order neural networks (HONNs) to forecast benchmark chaotic time series. Two models of HONNs were implemented, namely functional link neural network (FLNN) and pi-sigma neural network (PSNN). These models were tested on two benchmark time series; the monthly smoothed sunspot numbers and the Mackey-Glass time-delay differential equation time series. The forecasting performance of the HONNs is compared against the performance of different models previously used in the literature such as fuzzy and neural networks models. Simulation results showed that FLNN and PSNN offer good performance compared to many previously used hybrid models. Keywords— Chaotic time series; Sunspot time series; Mackey-Glass time series; higher order neural network; pi-sigma neural network; functional link neural network.