Transfer Entropy

Quantifying distributed information processing is crucial to understanding collective motion in animal groups. Recent studies have begun to apply rigorous methods based on information theory to quantify such distributed computation. Following this perspective, we use transfer entropy to quantify dynamic information flows locally in space and time across a school of fish during directional changes around a circular tank, i.e., U-turns. This analysis reveals peaks in information flows during collective U-turns and identifies two different flows: an informative flow (positive transfer entropy) from fish that have already turned to fish that are turning, and a misinformative flow (negative transfer entropy) from fish that have not turned yet to fish that are turning. We also reveal that the information flows are related to relative position and alignment between fish and identify spatial patterns of information and misinformation cascades. This study offers several methodological contributions and we expect further application of these methodologies to reveal intricacies of self-organisation in other animal groups and active matter in general.

In the framework of information dynamics, the temporal evolution of coupled systems can be studied by decomposing the predictive information about an assigned target system into amounts quantifying the information stored inside the system and the information transferred to it. While information storage and transfer are computed through the known self-entropy (SE) and transfer entropy (TE), an alternative decomposition evidences the so-called cross entropy (CE) and conditional SE (cSE), quantifying the cross information and internal information of the target system, respectively. This study presents a thorough evaluation of SE, TE, CE and cSE as quantities related to the causal statistical structure of coupled dynamic processes. First, we investigate the theoretical properties of these measures, providing the conditions for their existence and assessing the meaning of the information theoretic quantity that each of them reflects. Then, we present an approach for the exact computation of information dynamics based on the linear Gaussian approximation, and exploit this approach to characterize the behavior of SE, TE, CE and cSE in benchmark systems with known dynamics. Finally, we exploit these measures to study cardiorespiratory dynamics measured from healthy subjects during head-up tilt and paced breathing protocols. Our main result is that the combined evaluation of the measures of information dynamics allows to infer the causal effects associated with the observed

This paper deals with the information transfer mechanisms underlying causal relations between brain regions under resting condition. fMRI images of a large set of healthy individuals from the 1000 Functional Connectomes Beijing Zang dataset have been considered and the causal information transfer among brain regions studied using Transfer Entropy concepts. Thus, we explored the influence of a set of states in two given regions at time t (At Bt.) over the state of one of them at a following time step (Bt+1) and could observe a series of time-dependent events corresponding to four kinds of interactions, or causal rules, pointing to (de)activation and turn off mechanisms and sharing some features with positive and negative functional connectivity. The functional architecture emerging from such rules was modelled by a directional multilayer network based upon four interaction matrices and a set of indexes describing the effects of the network structure in several dynamical processes. Th...

The Kullback–Leibler (KL) divergence is a fundamental measure of information geometry that is used in a variety of contexts in artificial intelligence. We show that, when system dynamics are given by distributed nonlinear systems, this measure can be decomposed as a function of two information-theoretic measures, transfer entropy and stochastic interaction. More specifically, these measures are applicable when selecting a candidate model for a distributed system, where individual subsystems are coupled via latent variables and observed through a filter. We represent this model as a directed acyclic graph (DAG) that characterises the unidirectional coupling between subsystems. Standard approaches to structure learning are not applicable in this framework due to the hidden variables; however, we can exploit the properties of certain dynamical systems to formulate exact methods based on differential topology. We approach the problem by using reconstruction theorems to derive an analytical expression for the KL divergence of a candidate DAG from the observed dataset. Using this result, we present a scoring function based on transfer entropy to be used as a subroutine in a structure learning algorithm. We then demonstrate its use in recovering the structure of coupled Lorenz and Rössler systems.

Finding interdependency relations between (possibly multivariate) time series provides valuable knowledge about the processes that generate the signals. Information theory sets a natural framework for non-parametric measures of several classes of statistical dependencies. However, a reliable estimation from information-theoretic functionals is hampered when the dependency to be assessed is brief or evolves in time. Here, we show that these limitations can be overcome when we have access to an ensemble of independent repetitions of the time series. In particular, we gear a dataefficient estimator of probability densities to make use of the full structure of trial-based measures. By doing so, we can obtain time-resolved estimates for a family of entropy combinations (including mutual information, transfer entropy, and their conditional counterparts) which are more accurate than the simple average of individual estimates over trials. We show with simulated and real data that the proposed approach allows to recover the time-resolved dynamics of the coupling between different subsystems.

We develop networks of international stock market indices using information and correlation based measures. We use 83 stock market indices of a diversity of countries, as well as their single day lagged values, to probe the correlation and the flow of information from one stock index to another taking into account different operating hours. Additionally, we apply the formalism of partial correlations to build the dependency network of the data, and calculate the partial Transfer Entropy to quantify the indirect influence that indices have on one another. We find that Transfer Entropy is an effective way to quantify the flow of information between indices, and that a high degree of information flow between indices lagged by one day coincides to same day correlation between them.

This paper presents a data-driven pipeline for studying asymmetries in mutual interdependencies between distinct components of EEG signal. Due to volume conductance, estimating coherence between scalp electrodes may lead to spurious results. A group-based independent component analysis (ICA), which is conducted across all subjects and conditions simultaneously, is an alternative representation of the EEG measurements. Within this approach, the extracted components are independent in a global sense while short-lived or transient interdependencies may still be present between the components. In this paper, functional roles of the ICA components are specified through a partial least squares (PLS) analysis of task effects within the time course of the derived components. Functional integration is estimated within the information-theoretic approach using transfer entropy analysis based on asymmetries in mutual interdependencies of reconstructed phase dynamics. A secondary PLS analysis is performed to assess robust task-specific changes in transfer entropy estimates between functionally specific components.

Recent studies suggest that the functional organization of brain networks is altered in patients with severe disorders of consciousness (DOC), including coma [1]. A better characterization of these large-scale disturbances of brain function is fundamental for the understanding and eventual prediction of coma outcome. In this study we apply information theory tools and correlation measures to explore the brain functional connectivity (FC) in functional magnetic resonance imaging (fMRI) data from patients with DOC. In particular we are interested Figure 1 For each subject it is shown the PLC summed over the 45 regions of the left hemisphere (bottom graph), the right hemisphere (middle graph) or between all right-left homologous ROIs in the AAL (top graph). Numbers on x-axis are indicating the different DOC patient; the quotes, the second fMRI sessions. Mäki-Marttunen et al. BMC Neuroscience 2013, 14(Suppl 1):P83

The recurrent circuitry of the cerebral cortex generates an emergent pattern of activity that is organized into rhythmic periods of firing and silence referred to as slow oscillations (ca 1 Hz). Slow oscillations not only are dominant during slow wave sleep and deep anesthesia, but also can be generated by the isolated cortical network in vitro, being a sort of default activity of the cortical network. The cortex is densely and reciprocally connected with subcortical structures and, as a result, the slow oscillations in situ are the result of an interplay between cortex and thalamus. Due to this reciprocal connectivity and interplay, the mechanism responsible for the initiation of waves in the corticothalamocortical loop during slow oscillations is still a matter of debate. It was our objective to determine the directionality of the information flow between different layers of the cortex and the connected thalamus during spontaneous activity. With that purpose we obtained multilayer local field potentials from the rat visual cortex and from its connected thalamus, the lateral geniculate nucleus, during deep anaesthesia. We analyzed directionality of information flow between thalamus, cortical infragranular layers (5 and 6) and supragranular layers (2/3) by means of three information theoretical indicators: transfer entropy, symbolic transfer entropy and transcript mutual information. These three indicators coincided in finding that infragranular layers lead the information flow during slow oscillations both towards supragranular layers and towards the thalamus.

Causal relationships can often be found in visual object tracking between the motions of the camera and that of the tracked object. This object motion may be an effect of the camera motion, e.g. an unsteady handheld camera. But it may also be the cause, e.g. the cameraman framing the object. In this paper we explore these relationships, and provide statistical tools to detect and quantify them; these are based on transfer entropy and stem from information theory. The relationships are then exploited to make predictions about the object location. The approach is shown to be an excellent measure for describing such relationships. On the VOT2013 dataset the prediction accuracy is increased by 62% over the best non-causal predictor. We show that the location predictions are robust to camera shake and sudden motion, which is invaluable for any tracking algorithm and demonstrate this by applying causal prediction to two state-of-the-art trackers. Both of them benefit, Struck gaining a 7% accuracy and 22 % robustness increase on the VTB1.1 benchmark, becoming the new state-of-the-art.

Complexity of industrial plants and their stringent environmental and safety regulations have necessitated early detection and isolation of process faults. All the existing fault isolation methods can be categorized into two general groups: model-based and data-based. Transfer entropy is a data-based method for measuring propagation direction of disturbance and finding its root cause. In this paper, a new transfer entropy-based method is proposed to isolate different process faults. The novelty of this paper lies in using the transfer entropy idea to generate distinct patterns of information flow among process variables, recognize their correlations in the context of the transferred information in any abnormal condition, and finally isolate different process faults. The experimental results clearly demonstrate the superiority of the proposed method to the conventional methods.

Transfer entropy is a frequently employed measure of conditional co-dependence in non-parametric analysis of Granger causality. In this paper, we derive analytical expressions for transfer entropy for the multivariate exponential, logistic, Pareto (type I − IV) and Burr distributions. The latter two fall into the class of fat-tailed distributions with power law properties, used frequently in biological, physical and actuarial sciences. We discover that the transfer entropy expressions for all four distributions are identical and depend merely on the multivariate distribution parameter and the number of distribution dimensions. Moreover, we find that in all four cases the transfer entropies are given by the same decreasing function of distribution dimensionality.

In this study we investigated the olfactory coding mechanisms of the honey bee Apis mellifera by analyzing the dynamics of the antennal lobe glomeruli in response to the presentation of different olfactory stimuli. We produced functional and effective connectivity response networks for eight different stimuli and three specific temporal windows (ON, early-OFF, and OFF). The functional response graphs were produced from the zero-delay cross-correlation between the different glomerular signals, while the effective response graphs were based on the Granger causality of the same time series. When comparing the two measures, despite the fundamental differences in the concepts such as the presence of information on the directionality in Granger causality, we observed consistency between the correlation and causality results which show peculiar activation patterns related to the specific stimulus, specifically in the slow components of the signals. This suggests the possibility of integrating functional and effective connectivity to reduce false positive reports of statistically dependent patterns not resulting from physiological phenomena.