Assessing coupling dynamics from an ensemble of time series

Int. J. Bifurcation Chaos, 2004

ArXiv, 2021

We address in this study the problem of learning a summary causal graph on time series with potentially different sampling rates. To do so, we first propose a new temporal mutual information measure defined on a window-based representation of time series. We then show how this measure relates to an entropy reduction principle that can be seen as a special case of the Probabilistic Raising Principle. We finally combine these two ingredients in a PC-like algorithm to construct the summary causal graph. This algorithm is evaluated on several datasets that shows both its efficacy and efficiency.

Information-theoretic quantities, such as entropy and mutual information (MI), can be used to quantify the amount of information needed to describe a dataset or the information shared between two datasets. In the case of a dynamical system, the behavior of the relevant variables can be tightly coupled, such that information about one variable at a given instance in time may provide information about other variables at later instances in time. This is often viewed as a flow of information, and tracking such a flow can reveal relationships among the system variables. Since the MI is a symmetric quantity; an asymmetric quantity, called Transfer Entropy (TE), has been proposed to estimate the directionality of the coupling. However, accurate estimation of entropy-based measures is notoriously difficult. Every method has its own free tuning parameter(s) and there is no consensus on an optimal way of estimating the TE from a dataset. We propose a new methodology to estimate TE and apply a set of methods together as an accuracy cross-check to provide a reliable mathematical tool for any given data set. We demonstrate both the variability in TE estimation across techniques as well as the benefits of the proposed methodology to reliably estimate the directionality of coupling among variables.

Physical Review E, 2012

The analysis of dynamic dependencies in complex systems such as the brain helps to understand how emerging properties arise from interactions. Here we propose an information-theoretic framework to analyze the dynamic dependencies in multivariate time-evolving systems. This framework constitutes a fully multivariate extension and unification of previous approaches based on bivariate or conditional mutual information and Granger causality or transfer entropy. We define multi-information measures that allow us to study the global statistical structure of the system as a whole, the total dependence between subsystems, and the temporal statistical structure of each subsystem. We develop a stationary and a nonstationary formulation of the framework. We then examine different decompositions of these multi-information measures. The transfer entropy naturally appears as a term in some of these decompositions. This allows us to examine its properties not as an isolated measure of interdependence but in the context of the complete framework. More generally we use causal graphs to study the specificity and sensitivity of all the measures appearing in these decompositions to different sources of statistical dependence arising from the causal connections between the subsystems. We illustrate that there is no straightforward relation between the strength of specific connections and specific terms in the decompositions. Furthermore, causal and noncausal statistical dependencies are not separable. In particular, the transfer entropy can be nonmonotonic in dependence on the connectivity strength between subsystems and is also sensitive to internal changes of the subsystems, so it should not be interpreted as a measure of connectivity strength. Altogether, in comparison to an analysis based on single isolated measures of interdependence, this framework is more powerful to analyze emergent properties in multivariate systems and to characterize functionally relevant changes in the dynamics.

Computational Statistics & Data Analysis, 2019

A novel approach is proposed based on the weighted empirical likelihood to construct confidence intervals for dynamical correlation of random functions. The properties of the proposed confidence interval are investigated for random functions with regular or irregular observations. It is shown that the confidence interval using our new approach has a more accurate coverage probability than that using the traditional bootstrap method for random functions with irregular observations. Furthermore, simulation studies demonstrate that the new approach is considerably more efficient in computation than the bootstrap method. The new approach is illustrated with three applications. The first application investigates the dynamical correlation of air pollutants. The second application studies the dynamical correlation of EEG signals in different regions of the brain in response to some stimuli. The third application estimates the dynamical correlation of gene expressions during the activation of T-cells.

Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference, 2010

This study introduces a new approach for the detection of nonlinear Granger causality between dynamical systems. The approach is based on embedding the multivariate (MV) time series measured from the systems X and Y by means of a sequential, non-uniform procedure, and on using the corrected conditional entropy (CCE) as unpredictability measure. The causal coupling from X to Y is quantified as the relative decrease of CCE measured after allowing the series of X to enter the embedding procedure for the description of Y. The ability of the approach to quantify nonlinear causality is assessed on MV time series measured from simulated dynamical systems with unidirectional coupling (the Rössler-Lorenz deterministic system) and bidirectional coupling (two coupled stochastic systems). The method is then applied to real magnetoencephalographic data measured during a visuo-tactile cognitive experiment, showing values of causal coupling consistent with the hypothesis of a cross-processing of d...

arXiv (Cornell University), 2021

Mutual information is a widely-used information theoretic measure to quantify the amount of association between variables. It is used extensively in many applications such as image registration, diagnosis of failures in electrical machines, pattern recognition, data mining and tests of independence. The main goal of this paper is to provide an efficient estimator of the mutual information based on the approach of Al Labadi et. al. (2021). The estimator is explored through various examples and is compared to its frequentist counterpart due to Berrett et al. (2019). The results show the good performance of the procedure by having a smaller mean squared error.

2007

In nonlinear systems, the understanding of underlying nonlinear processes and their interactions is very important for predictive modeling as well as for generating bounds on predictability. However, data analysis methods based on nonlinear dynamical approaches are typically not robust when applied to short and noisy data 1. The definition of what constitutes short and noisy, in terms of data sizes and noiseto-signal ratios, may be application and context specific.

Physica D: Nonlinear Phenomena, 2014

Inference of causality is central in nonlinear time series analysis and science in general. A popular approach to infer causality between two processes is to measure the information flow between them in terms of transfer entropy. Using dynamics of coupled oscillator networks, we show that although transfer entropy can successfully detect information flow in two processes, it often results in erroneous identification of network connections under the presence of indirect interactions, dominance of neighbors, or anticipatory couplings. Such effects are found to be profound for time-dependent networks. To overcome these limitations, we develop a measure called causation entropy, and show that its application can lead to reliable identification of true couplings.

Entropy

We introduce an information-theoretical approach for analyzing information transfer between time series. Rather than using the Transfer Entropy (TE), we define and apply the Transfer Information Energy (TIE), which is based on Onicescu's Information Energy. Whereas the TE can be used as a measure of the reduction in uncertainty about one time series given another, the TIE may be viewed as a measure of the increase in certainty about one time series given another. We compare the TIE and the TE in two known time series prediction applications. First, we analyze stock market indexes from the Americas, Asia/Pacific and Europe, with the goal to infer the information transfer between them (i.e., how they influence each other). In the second application, we take a bivariate time series of the breath rate and instantaneous heart rate of a sleeping human suffering from sleep apnea, with the goal to determine the information transfer heart → breath vs. breath → heart. In both applications, the computed TE and TIE values are strongly correlated, meaning that the TIE can substitute the TE for such applications, even if they measure symmetric phenomena. The advantage of using the TIE is computational: we can obtain similar results, but faster.