Information flow and optimization in transcriptional regulation

Nature Genetics, 2011

A gene's activity can be described by the discrete time series of mRNA production events 1,2 . This transcriptional time series is stochastic rather than deterministic 2-4 . Furthermore, it generally cannot be described as a simple Poisson process. In other words, mRNA molecules are not produced with a constant probability per unit time; instead, mRNA production is often bursty (pulsatile) in both bacteria 2 and higher organisms 4-8 . A suitable mathematical framework for describing gene activity data is the two-state model 8-10 , where a gene stochastically fluctuates between 'off ' and 'on' states, and mRNA is produced stochastically only in the on state. This scenario can lead to the occurrence of transcription 'bursts' , periods of intense activity separated by periods of quiescence. Measured mRNA kinetics 2,5 and copy-number statistics have been shown to be consistent with the two-state picture in a variety of model systems. However, despite considerable theoretical attention 2,13-17 , we do not have a biophysical understanding of the nature of the on and off states and what governs the transitions between them.

Physical Review E, 2012

Living cells must control the reading out or "expression" of information encoded in their genomes, and this regulation often is mediated by transcription factors-proteins that bind to DNA and either enhance or repress the expression of nearby genes. But the expression of transcription factor proteins is itself regulated, and many transcription factors regulate their own expression in addition to responding to other input signals. Here we analyze the simplest of such self-regulatory circuits, asking how parameters can be chosen to optimize information transmission from inputs to outputs in the steady state. Some nonzero level of self-regulation is almost always optimal, with selfactivation dominant when transcription factor concentrations are low and self-repression dominant when concentrations are high. In steady-state the optimal self-activation is never strong enough to induce bistability, although there is a limit in which the optimal parameters are very close to the critical point. arXiv:1112.5026v1 [q-bio.MN]

Nature Communications

Self-organization is the generation of order out of local interactions. It is deeply connected to many fields of science from physics, chemistry to biology, all based on physical interactions. The emergence of collective animal behavior is the result of self-organization processes as well, though they involve abstract interactions arising from sensory inputs, information processing, storage, and feedback. Resulting collective behaviors are found, for example, in crowds of people, flocks of birds, and swarms of bacteria. Here we introduce interactions between active microparticles which are based on the information about other particle positions. A real-time feedback of multiple active particle positions is the information source for the propulsion direction of these particles. The emerging structures require continuous information flows. They reveal frustrated geometries due to confinement to two dimensions and internal dynamical degrees of freedom that are reminiscent of physically bound systems, though they exist only as nonequilibrium structures.

Entropy

The entropy production in stochastic dynamical systems is linked to the structure of their causal representation in terms of Bayesian networks. Such a connection was formalized for bipartite (or multipartite) systems with an integral fluctuation theorem in [Phys. Rev. Lett. 111, 180603 (2013)]. Here we introduce the information thermodynamics for time series, that are non-bipartite in general, and we show that the link between irreversibility and information can only result from an incomplete causal representation. In particular, we consider a backward transfer entropy lower bound to the conditional time series irreversibility that is induced by the absence of feedback in signal-response models. We study such a relation in a linear signal-response model providing analytical solutions, and in a nonlinear biological model of receptor-ligand systems where the time series irreversibility measures the signaling efficiency.

The promoter state of a gene and its expression levels are modulated by the amounts of transcription factors interacting with its regulatory regions. Hence, one may interpret a gene network as a communicating system in which the state of the promoter of a gene (the source) is communicated by the amounts of transcription factors that it expresses (the message) to modulate the state of the promoter and expression levels of another gene (the receptor). The reliability of the gene network dynamics can be quantified by the Shannon’s entropy of the message and the mutual information between the message and the promoter state. Here we consider a stochastic model for a binary gene and use its exact steady state solutions to calculate the entropy and mutual information. We show that a slow switching promoter having long and equally standing ON and OFF states maximizes the mutual information and reduces entropy. That is a bursty regime generating a high variance message governed by a bimodal ...

Entropy

The promoter state of a gene and its expression levels are modulated by the amounts of transcription factors interacting with its regulatory regions. Hence, one may interpret a gene network as a communicating system in which the state of the promoter of a gene (the source) is communicated by the amounts of transcription factors that it expresses (the message) to modulate the state of the promoter and expression levels of another gene (the receptor). The reliability of the gene network dynamics can be quantified by Shannon’s entropy of the message and the mutual information between the message and the promoter state. Here we consider a stochastic model for a binary gene and use its exact steady state solutions to calculate the entropy and mutual information. We show that a slow switching promoter with long and equally standing ON and OFF states maximizes the mutual information and reduces entropy. That is a binary gene expression regime generating a high variance message governed by ...

Entropy

In order to respond to environmental signals, cells often use small molecular circuits to transmit information about their surroundings. Recently, motivated by specific examples in signaling and gene regulation, a body of work has focused on the properties of circuits that function out of equilibrium and dissipate energy. We briefly review the probabilistic measures of information and dissipation and use simple models to discuss and illustrate trade-offs between information and dissipation in biological circuits. We find that circuits with non-steady state initial conditions can transmit more information at small readout delays than steady state circuits. The dissipative cost of this additional information proves marginal compared to the steady state dissipation. Feedback does not significantly increase the transmitted information for out of steady state circuits but does decrease dissipative costs. Lastly, we discuss the case of bursty gene regulatory circuits that, even in the fas...

Proceedings of the National Academy of Sciences, 2009

PLOS Computational Biology, 2015

Cell signaling dynamics and transcriptional regulatory activities are variable within specific cell types responding to an identical stimulus. In addition to studying the network interactions, there is much interest in utilizing single cell scale data to elucidate the non-random aspects of the variability involved in cellular decision making. Previous studies have considered the information transfer between the signaling and transcriptional domains based on an instantaneous relationship between the molecular activities. These studies predict a limited binary on/off encoding mechanism which underestimates the complexity of biological information processing, and hence the utility of single cell resolution data. Here we pursue a novel strategy that reformulates the information transfer problem as involving dynamic features of signaling rather than molecular abundances. We pursue a computational approach to test if and how the transcriptional regulatory activity patterns can be informative of the temporal history of signaling. Our analysis reveals (1) the dynamic features of signaling that significantly alter transcriptional regulatory patterns (encoding), and (2) the temporal history of signaling that can be inferred from single cell scale snapshots of transcriptional activity (decoding). Immediate early gene expression patterns were informative of signaling peak retention kinetics, whereas transcription factor activity patterns were informative of activation and deactivation kinetics of signaling. Moreover, the information processing aspects varied across the network, with each component encoding a selective subset of the dynamic signaling features. We developed novel sensitivity and information transfer maps to unravel the dynamic multiplexing of signaling features at each of these network components. Unsupervised clustering of the maps revealed two groups that aligned with network motifs distinguished by transcriptional feedforward vs feedback interactions. Our new computational methodology impacts the single cell scale experiments by identifying downstream snapshot measures required for inferring specific dynamical features of upstream signals involved in the regulation of cellular responses.

Neural Networks, 2021

Spatial information transfer in hippocampal place cells depends on trial-to-trial variability, symmetry of place-field firing, and biophysical heterogeneities. Neural Networks (2021), doi:

Physical Review E, 2017

The intuition of causation is so fundamental that almost every research study in life sciences refers to this concept. However a widely accepted formal definition of causal influence between observables is still missing. In the framework of linear Langevin networks without feedbacks (linear response models) we developed a measure of causal influence based on a decomposition of information flows over time. We discuss its main properties and compare it with other information measures like the Transfer Entropy. Finally we outline some difficulties of the extension to a general definition of causal influence for complex systems. "The causes of all the appearances in nature are the conditions under which they reliably emerge".

Artificial Life, 2019

In nature, gene regulatory networks are a key mediator between the information stored in the DNA of living organisms (their genotype) and the structural and behavioral expression this finds in their bodies, surviving in the world (their phenotype). They integrate environmental signals, steer development, buffer stochasticity, and allow evolution to proceed. In engineering, modeling and implementations of artificial gene regulatory networks have been an expanding field of research and development over the past few decades. This review discusses the concept of gene regulation, describes the current state of the art in gene regulatory networks, including modeling and simulation, and reviews their use in artificial evolutionary settings. We provide evidence for the benefits of this concept in natural and the engineering domains.

Cell Systems, 2020

Highlights d p53 oscillations lead to oscillations in p21 transcription and p21 protein derivative d p53 levels affect the probability, rather than the magnitude, of p21 transcription d Independent p21 alleles within a cell show highly correlated transcriptional dynamics d Oscillations of p53 levels and p21 transcription dynamics dampen p21 protein levels

Physical Review E, 2016

We present a stochastic framework to study signal transmission in a generic two-step cascade S → X → Y. Starting from a set of Langevin equations obeying Gaussian noise processes we calculate the variance and covariance while considering both linear and nonlinear production terms for different biochemical species of the cascade. These quantities are then used to calculate the net synergy within the purview of partial information decomposition. We show that redundancy in information transmission is essentially an important consequence of Markovian property of the two-step cascade motif. We also show that redundancy increases fidelity of the signalling pathway.

Chaos: An Interdisciplinary Journal of Nonlinear Science, 2018

The formalism of partial information decomposition provides a number of independent components which altogether constitute the total information provided by the source variable(s) about the target variable(s). These non-overlapping terms are recognized as unique information, synergistic information and, redundant information. The metric of net synergy conceived as the difference between synergistic and redundant information, is capable of detecting effective synergy, effective redundancy and, information independence among stochastic variables. The net synergy can be quantified using appropriate combinations of different Shannon mutual information terms. The utilization of the net synergy in network motifs with the nodes representing different biochemical species, involved in information sharing, uncovers rich store for exciting results. In the current study, we use this formalism to obtain a comprehensive understanding of the relative information processing mechanism in a diamond motif and two of its sub-motifs namely bifurcation and integration motif embedded within the diamond motif. The emerging patterns of effective synergy and effective redundancy and their contribution towards ensuring high fidelity information transmission are duly compared in the submotifs. Investigation on the metric of net synergy in independent bifurcation and integration motifs are also executed. In all of these computations, the crucial roles played by various systemic time scales, activation coefficients and signal integration mechanisms at the output of the network topologies are especially emphasized. Following this plan of action, we become confident that the origin of effective synergy and effective redundancy can be architecturally justified by decomposing a diamond motif into bifurcation and integration motif. According to our conjecture, the presence of common source of fluctuations creates effective redundancy. Our calculations reveal effective redundancy empowers signal fidelity. Moreover, to achieve this, input signaling species avoids strong interaction with downstream intermediates. This strategy is capable of making the diamond motif noise-tolerant. Apart from the topological features, our study also puts forward the active contribution of additive and multiplicative signal integration mechanisms to nurture effective redundancy and effective synergy.

Theory in Biosciences, 2021

We present an information-theoretic formalism to study signal transduction in four architectural variants of a model two-step cascade with increasing input population. Our results categorize these four types into two classes depending upon the effect played out by activation and repression on mutual information, net synergy, and signal-to-noise ratio. Within the Gaussian framework and using the linear noise approximation, we derive the analytic expressions for these metrics to establish their underlying relationships in terms of the biochemical parameters. We also verify our approximations through stochastic simulations.

Physical Review E, 2017

We use information theory to study the information transmission through a simple gene cascade where the product of an unregulated gene regulates the expression activity of a cooperative genetic switch. While the input signal is provided by the upstream gene with two states, we consider that the expression of downstream gene is controlled by a cis-regulatory system with three binding sites for the regulator product, which can bind cooperatively. By computing exactly the associated probability distributions, we estimate information transmission thought the mutual information measure. We found that the mutual information associated with unimodal input signal is lower than the associated with bimodal inputs. We also observe that mutual information presents a maximum in the cooperativity intensity, and the position of this maximum depends on the kinetic rates of the promoter. Furthermore, we found that the bursting dynamics of the input signal can enhance the information transmission capacity.

2020 IEEE International Symposium on Information Theory (ISIT)

A minimal model for gene expression, consisting of a switchable promoter together with the resulting messenger RNA, is equivalent to a Poisson channel with a binary Markovian input process. Determining its capacity is an optimization problem with respect to two parameters: the average sojourn times of the promoter's active (ON) and inactive (OFF) state. An expression for the mutual information is found by solving the associated filtering problem analytically on the level of distributions. For fixed peak power, three bandwidth-like constraints are imposed by lower-bounding (i) the average sojourn times (ii) the autocorrelation time and (iii) the average time until a transition. OFFfavoring optima are found for all three constraints, as commonly encountered for the Poisson channel. In addition, constraint (i) exhibits a region that favors the ON state, and (iii) shows ONfavoring local optima.

Physical Review E, 2010

The identification of motifs-subgraphs that appear significantly more often in a particular network than in an ensemble of randomized networks-has become a ubiquitous method for uncovering potentially important subunits within networks drawn from a wide variety of fields. We find that the most common algorithms used to generate the ensemble from the real network change subgraph counts in a highly correlated manner, so that one subgraph's status as a motif may not be independent from the statuses of the other subgraphs. We demonstrate this effect for the problem of 3-and 4-node motif identification in the transcriptional regulatory networks of E. coli and S. cerevisiae in which randomized networks are generated via an edge-swapping algorithm (Milo et al., Science 298:824, 2002). We show that correlations among 3-node subgraphs are easily interpreted, and we present an information-theoretic tool that may be used to identify correlations among subgraphs of any size.

Scientific Reports

Cells detect changes in their environment and generate responses, often involving changes in gene expression. In this paper we use information theory and a simple transcription model to analyze whether the resulting gene expression serves to identify extracellular stimuli and assess their intensity when they are encoded in the amplitude, duration or frequency of pulses of a transcription factor’s nuclear concentration (or activation state). We find, for all cases, that about three ranges of input strengths can be distinguished and that maximum information transmission occurs for fast and high activation threshold promoters. The three input modulation modes differ in the sensitivity to changes in the promoters parameters. Frequency modulation is the most sensitive and duration modulation, the least. This is key for signal identification: there are promoter parameters that yield a relatively high information transmission for duration or amplitude modulation and a much smaller value fo...

2019

The single-input module (SIM) is a regulatory motif capable of coordinating gene expression across functionally related genes. We explore the relationship between regulation of the central autoregulated TF in a negatively regulated SIM and the target genes using a synthetic biology approach paired with stochastic simulations. Surprisingly, we find a fundamental asymmetry in the level of regulation experienced by the TF gene and its targets, even if they have identical regulatory DNA; the TF gene experiences stronger repression than its targets. This asymmetry is not predicted from deterministic modeling of the system but is revealed from corresponding stochastic simulations. The magnitude of asymmetry depends on factors such as the number of targets in the SIM, TF degradation rate (or growth rate) and TF binding affinity. Beyond implications for SIM motifs, the influence of network connectivity on regulatory levels highlights an interesting challenge for predictive models of gene re...

eLife, 2020

Predicting gene expression from DNA sequence remains a major goal in the field of gene regulation. A challenge to this goal is the connectivity of the network, whose role in altering gene expression remains unclear. Here, we study a common autoregulatory network motif, the negative single-input module, to explore the regulatory properties inherited from the motif. Using stochastic simulations and a synthetic biology approach in E. coli, we find that the TF gene and its target genes have inherent asymmetry in regulation, even when their promoters are identical; the TF gene being more repressed than its targets. The magnitude of asymmetry depends on network features such as network size and TF-binding affinities. Intriguingly, asymmetry disappears when the growth rate is too fast or too slow and is most significant for typical growth conditions. These results highlight the importance of accounting for network architecture in quantitative models of gene expression.

Physical Review E

Adaptation is used by biological sensory systems to respond to a wide range of environmental signals, by adapting their response properties to the statistics of the stimulus in order to maximize information transmission. We derive rules of optimal adaptation to changes in the mean and variance of a continuous stimulus in terms of Bayesian filters, and map them onto stochastic equations that couple the state of the environment to an internal variable controling the response function. We calculate numerical and exact results for the speed and accuracy of adaptation, and its impact on information transmission. We find that, in the regime of efficient adaptation, the speed of adaptation scales sublinearly with the rate of change of the environment. Finally, we exploit the mathematical equivalence between adaptation and stochastic thermodynamics to quantitatively relate adaptation to the irreversibility of the adaptation time course, defined by the rate of entropy production. Our results suggest a means to empirically quantify adaptation in a model-free and non-parametric way.

PLoS Computational Biology, 2014

Sensory systems have evolved to respond to input stimuli of certain statistical properties, and to reliably transmit this information through biochemical pathways. Hence, for an experimentally well-characterized sensory system, one ought to be able to extract valuable information about the statistics of the stimuli. Based on dose-response curves from in vivo fluorescence resonance energy transfer (FRET) experiments of the bacterial chemotaxis sensory system, we predict the chemical gradients chemotactic Escherichia coli cells typically encounter in their natural environment. To predict average gradients cells experience, we revaluate the phenomenological Weber's law and its generalizations to the Weber-Fechner law and fold-change detection. To obtain full distributions of gradients we use information theory and simulations, considering limitations of information transmission from both cell-external and internal noise. We identify broad distributions of exponential gradients, which lead to log-normal stimuli and maximal drift velocity. Our results thus provide a first step towards deciphering the chemical nature of complex, experimentally inaccessible cellular microenvironments, such as the human intestine.

Scientific Reports, 2019

Cells are often considered input-output devices that maximize the transmission of information by converting extracellular stimuli (input) via signaling pathways (communication channel) to cell behavior (output). However, in biological systems outputs might feed back into inputs due to cell motility, and the biological channel can change by mutations during evolution. Here, we show that the conventional channel capacity obtained by optimizing the input distribution for a fixed channel may not reflect the global optimum. In a new approach we analytically identify both input distributions and input-output curves that optimally transmit information, given constraints from noise and the dynamic range of the channel. We find a universal optimal input distribution only depending on the input noise, and we generalize our formalism to multiple outputs (or inputs). Applying our formalism to Escherichia coli chemotaxis, we find that its pathway is compatible with optimal information transmissi...

Journal of Statistical Physics, 2011

High throughput experiments, characteristic of studies in systems biology, produce large output data sets often at different time points or under a variety of related conditions or for different patients. In several recent papers the data is modeled by using a distribution of maximal information-theoretic entropy. We pose the question: 'whose entropy' meaning how do we select the variables whose distribution should be compared to that of maximal entropy. The point is that different choices can lead to different answers. Due to the technological advances that allow for the system-wide measurement of hundreds to thousands of events from biological samples, addressing this question is now part of the analysis of systems biology datasets. The analysis of the extent of phosphorylation in reference to the transformation potency of Bcr-Abl fusion oncogene mutants is used as a biological example. The approach taken seeks to use entropy not simply as a statistical measure of dispersion but as a physical, thermodynamic, state function. This highlights the dilemma of what are the variables that describe the state of the signaling network. Is what matters Boolean, spin-like, variables that specify whether a particular phosphorylation site is or is not actually phosphorylated. Or does the actual extent of phosphorylation matter. Last but not least is the possibility that in a signaling network some few specific phosphorylation sites are the key to the signal transduction even though these sites are not at any time abundantly phosphorylated in an absolute sense.