Forecasting chaotic time series: Global vs. local methods

Physica D: Nonlinear Phenomena, 1998

Local linear prediction, based on the ordinary least squares (OLS) approach, is one of several methods that have been applied to prediction of chaotic time series. Apart from potential linearization errors, a drawback of this approach is the high variance of the predictions under certain conditions. Here, a different set of so-called linear regularization techniques, originally derived to solve ill-posed regression problems, are compared to OLS for chaotic time series corrupted by additive measurement noise. These methods reduce the variance compared to OLS, but introduce more bias. A main tool of analysis is the singular value decomposition (SVD), and a key to successful regularization is to damp the higher order SVD components. Several of the methods achieve improved prediction compared to OLS for synthetic noise-corrupted data from well-known chaotic systems. Similar results were found for real-world data from the R-R intervals of ECG signals. Good results are also obtained for real sunspot data, compared to published predictions using nonlinear techniques.

2013

We propose a novel methodology for forecasting chaotic systems which is based on exploiting the information conveyed by the local Lyapunov exponents of a system. This information is used to correct for the inevitable bias of most non-parametric predictors. Using simulated data, we show that gains in prediction accuracy can be substantial.

INTERNATIONAL …, 2012

In this paper, we present a new chaotic control algorithm that leads to a fairly substantial improvement in the performance of the traditional fuzzy chaotic time series forecaster. The main idea is to use the Fuzzy Basis Functions (FBFs) expansion which allows us to view a Fuzzy Logic System (FLS) as a linear combination of the consequent parameters. Hence, the possibility to use a linear optimization to tune these parameters, which results in a two-stage algorithm. The first step consists in extracting the training data from a chaotic system with different initial conditions, and the second one consists on applying the generalized orthogonality principle, in order to construct a globally more efficient FLS forecaster. Such a forecaster yields the ability to produce substantially improved results when applied to predict the output of the Mackey-Glass time series.

Applied Mathematics and Computation, 1998

This report was prepared as an account of work sponsored by an agency of the United States Government Neither the United States Government nor any agency themf, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or proctss disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or servia by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, m mmendotion, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof. DISCLAIMER Portions of this document may be illegible in electronic image products. Images are produced from the best available original document.

Physical Review Letters, 2000

Chaos: An Interdisciplinary Journal of Nonlinear Science, 2018

A model-based approach to forecasting chaotic dynamical systems utilizes knowledge of the physical processes governing the dynamics to build an approximate mathematical model of the system. In contrast, machine learning techniques have demonstrated promising results for forecasting chaotic systems purely from past time series measurements of system state variables (training data), without prior knowledge of the system dynamics. The motivation for this paper is the potential of machine learning for filling in the gaps in our underlying mechanistic knowledge that cause widely-used knowledge-based models to be inaccurate. Thus we here propose a general method that leverages the advantages of these two approaches by combining a knowledge-based model and a machine learning technique to build a hybrid forecasting scheme. Potential applications for such an approach are numerous (e.g., improving weather forecasting). We demonstrate and test the utility of this approach using a particular illustrative version of a machine learning known as reservoir computing, and we apply the resulting hybrid forecaster to a low-dimensional chaotic system, as well as to a high-dimensional spatiotemporal chaotic system. These tests yield extremely promising results in that our hybrid technique is able to accurately predict for a much longer period of time than either its machine-learning component or its model-based component alone.

Neural Processing …, 2011

The accuracy of a model to forecast a time series diminishes as the prediction horizon increases, in particular when the prediction is carried out recursively. Such decay is faster when the model is built using data generated by highly dynamic or chaotic systems. This paper presents a topology and training scheme for a novel artificial neural network, named "Hybrid-connected Complex Neural Network" (HCNN), which is able to capture the dynamics embedded in chaotic time series and to predict long horizons of such series. HCNN is composed of small recurrent neural networks, inserted in a structure made of feed-forward and recurrent connections and trained in several stages using the algorithm back-propagation through time (BPTT). In experiments using a Mackey-Glass time series and an electrocardiogram (ECG) as training signals, HCNN was able to output stable chaotic signals, oscillating for periods as long as four times the size of the training signals. The largest local Lyapunov Exponent (LE) of predicted signals was positive (an evidence of chaos), and similar to the LE calculated over the training signals. The magnitudes of peaks in the ECG signal were not accurately predicted, but the predicted signal was similar to the ECG in the rest of its structure.

Complexity

In this article, two models of the forecast of time series obtained from the chaotic dynamic systems are presented: the Lorenz system, the manufacture system, and the volume of the Great Salt Lake of Utah. The theory of the nonlinear dynamic systems indicates the capacity of making good-quality predictions of series coming from dynamic systems with chaotic behavior up to a temporal horizon determined by the inverse of the major Lyapunov exponent. The analysis of the Fourier power spectrum and the calculation of the maximum Lyapunov exponent allow confirming the origin of the series from a chaotic dynamic system. The delay time and the global dimension are employed as parameters in the models of forecast of artificial neuronal networks (ANN) and support vector machine (SVM). This research demonstrates how forecast models built with ANN and SVM have the capacity of making forecasts of good quality, in a superior temporal horizon at the determined interval by the inverse of the maximum...

Echo State Networks (ESN) present a novel approach to analysing and training recurrent neural networks (RNNs). It leads to a fast, simple and constructive algorithm for supervised training of RNNs. A very powerful blackbox modeling tool to build models to simulate, predict, filter, classify, or control nonlinear dynamical systems, what makes ESNs excel over traditional techniques is that it can efficiently encode and retain massive information in an ESN "echo" network state about a long previous history. This makes ESNs excellent approximators of, among other nonlinear dynamical systems, chaotic time series -with obvious applications in prediction of such tasks as currency exchange rates. This research proposal aims to further improve upon the best empirical result of predicting a chaotic time series, which was obtained by an ESN, by replacing the linear readout mechanism employed by ESNs with a multi layer perceptron (MLP). This shall allow the resulting network to harness the dynamical memory of an ESN with the approximating powers of the gradient descent algorithm of a MLP, resulting in a powerful approximator, smaller in size than using just an ESN and allowing for more feasible practical implementations in telecommunications.

2008

This paper examines how efficient neural networks are relative to linear and polynomial approximations to forecast a time series that is generated by the chaotic Mackey-Glass differential delay equation. The forecasting horizon is one step ahead. A series of regressions with polynomial approximators and a simple neural network with two neurons is taking place and compare the multiple correlation coefficients. The neural network, a very simple neural network, is superior to the polynomial expansions, and delivers a virtually perfect forecasting. Finally, the neural network is much more precise, relative to the other methods, across a wide set of realizations.