Academia.eduAcademia.edu

Outline

Title
Abstract
References
All Topics
Computer Science
Machine Learning

Estimating time-varying networks

Profile image of Amr AhmedAmr Ahmed

2008

https://doi.org/10.1214/09-AOAS308
visibility

…

description

32 pages

link

1 file

Abstract

Stochastic networks are a plausible representation of the relational information among entities in dynamic systems such as living cells or social communities. While there is a rich literature in estimating a static or temporally invariant network from observation data, little has been done toward estimating time-varying networks from time series of entity attributes. In this paper we present two new machine learning methods for estimating time-varying networks, which both build on a temporally smoothed l1-regularized logistic regression formalism that can be cast as a standard convex-optimization problem and solved efficiently using generic solvers scalable to large networks. We report promising results on recovering simulated timevarying networks. For real data sets, we reverse engineer the latent sequence of temporally rewiring political networks between Senators from the US Senate voting records and the latent evolving regulatory networks underlying 588 genes across the life cycle of Drosophila melanogaster from the microarray time course. . This reprint differs from the original in pagination and typographic detail. 1 2 KOLAR, SONG, AHMED AND XING Each of these characteristics adds a degree of complexity to the interpretation and analysis of networks. In this paper we present a new methodology and analysis that address a particular aspect of dynamic network analysis: how can one reverse engineer networks that are latent, and topologically evolving over time, from time series of nodal attributes. While there is a rich and growing literature on modeling time-invariant networks, much less has been done toward modeling dynamic networks that are rewiring over time. We refer to these time or condition specific circuitries as time-varying networks, which are ubiquitous in various complex systems. Consider the following two real world problems:

Related papers

Models for Capturing Temporal Smoothness in Evolving Networks for Learning Latent Representation of Nodes
Mohammad Al Hasan

2018

In a dynamic network, the neighborhood of the vertices evolve across different temporal snapshots of the network. Accurate modeling of this temporal evolution can help solve complex tasks involving real-life social and interaction networks. However, existing models for learning latent representation are inadequate for obtaining the representation vectors of the vertices for different time-stamps of a dynamic network in a meaningful way. In this paper, we propose latent representation learning models for dynamic networks which overcome the above limitation by considering two different kinds of temporal smoothness: (i) retrofitted, and (ii) linear transformation. The retrofitted model tracks the representation vector of a vertex over time, facilitating vertex-based temporal analysis of a network. On the other hand, linear transformation based model provides a smooth transition operator which maps the representation vectors of all vertices from one temporal snapshot to the next (unobserved) snapshot-this facilitates prediction of the state of a network in a future time-stamp. We validate the performance of our proposed models by employing them for solving the temporal link prediction task. Experiments on 9 real-life networks from various domains validate that the proposed models are significantly better than the existing models for predicting the dynamics of an evolving network.

View PDFchevron_right
Semi-parametric Methods for Estimating Time-varying Graph Structure
Mladen Kolar

2010

Abstract Stochastic networks are a plausible representation of the relational information among entities in dynamic systems such as living cells or social communities. While there is a rich literature in estimating a static or temporally invariant network from observation data, little has been done towards estimating time-varying networks from time series of entity attributes.

View PDFchevron_right
Statistical inference of the time-varying structure of gene-regulation networks
Michael P.H. Stumpf

BMC systems biology, 2010

Background: Biological networks are highly dynamic in response to environmental and physiological cues. This variability is in contrast to conventional analyses of biological networks, which have overwhelmingly employed static graph models which stay constant over time to describe biological systems and their underlying molecular interactions.

View PDFchevron_right
A statistical framework for recovering pseudo-dynamic networks from static data
Tianzhou Ma

Bioinformatics, 2022

Motivation The collection of temporal or perturbed data is often a prerequisite for reconstructing dynamic networks in most cases. However, these types of data are seldom available for genomic studies in medicine, thus significantly limiting the use of dynamic networks to characterize the biological principles underlying human health and diseases. Results We proposed a statistical framework to recover disease risk-associated pseudo-dynamic networks (DRDNet) from steady-state data. We incorporated a varying coefficient model with multiple ordinary differential equations to learn a series of networks. We analyzed the publicly available Genotype-Tissue Expression data to construct networks associated with hypertension risk, and biological findings showed that key genes constituting these networks had pivotal and biologically relevant roles associated with the vascular system. We also provided the selection consistency of the proposed learning procedure and evaluated its utility through...

View PDFchevron_right
Recovering time-varying networks of dependencies in social and biological studies
Amr Ahmed

Proceedings of The National Academy of Sciences, 2009

Author contributions: E.P.X. designed research; A.A. and E.P.X. performed research; A.A. and E.P.X. contributed new reagents/analytic tools; A.A. and E.P.X. analyzed data; and E.P.X. and A.A. wrote the paper.

View PDFchevron_right
Modelling slowly changing dynamic gene-regulatory networks
A. Abbruzzo

Dynamic gene-regulatory networks are complex since the number of potential components involved in the system is very large. Estimating dynamic networks is an important task because they compromise valuable information about interactions among genes. Graphical models are a powerful class of models to estimate conditional independence among random variables, e.g. interactions in dynamic systems. Indeed, these interactions tend to vary over time. However, the literature has been focused on static networks, which can only reveal overall structures. Time-course experiments are performed in order to tease out significant changes in networks. It is typically reasonable to assume that changes in genomic networks are few because systems in biology tend to be stable. We introduce a new model for estimating slowly changes in dynamic gene-regulatory networks which is suitable for a highdimensional dataset, e.g. time-course genomic data. Our method is based on i) the penalized likelihood with ℓ 1 -norm, ii) the penalized differences between conditional independence elements across time points and iii) the heuristic search strategy to find optimal smoothing parameters. We implement a set of linear constraints necessary to estimate sparse graphs and penalized changing in dynamic networks. These constraints are not in the linear form. For this reason, we introduce slack variables to re-write our problem into a standard convex optimization problem subject to equality linear constraints. We show that GL ∆ performs well in a simulation study. Finally, we apply the proposed model to a time-course genetic dataset T-cell.

View PDFchevron_right
Network Reconstruction from Perturbation Time Course Data
Deepraj Sarmah

SummaryQuantitative, directional network structure inference remains challenging even for small systems, particularly when loops and cycles are present. We report a method that robustly infers direct, signed connections between network nodes from noisy, sparse perturbation time course data requiring only one perturbation per node. We find good sensitivity and specificity for classification, as well as quantitative agreement in randomized 2- and 3-node systems having varied and complex dynamics. Experimental application of the method to the ERK and AKT pathways, widely important in mammalian signaling, reveals evidence of bi-directional cross-talk coupled with strong negative feedback on both pathways, consistent with prior knowledge. Systematic application of this method can reduce important subnetwork structural uncertainty, enabling better prediction of dynamics, response to perturbations such as drugs, and understanding of biological networks. The method is general and can be app...

View PDFchevron_right
Identifying Evolving Groups in Dynamic Multimode Networks
Huan Liu

IEEE Transactions on Knowledge and Data Engineering, 2012

A multi-mode network consists of heterogeneous types of actors with various interactions occurring between them. Identifying communities in a multi-mode network can help understand the structural properties of the network, address the data shortage and unbalanced problems, and assist tasks like targeted marketing and finding influential actors within or between groups. In general, a network and its group structure often evolve unevenly. In a dynamic multimode network, both group membership and interactions can evolve, posing a challenging problem of identifying these evolving communities. In this work, we try to address this problem by employing the temporal information to analyze a multi-mode network. A temporally-regularized framework and its convergence property are carefully studied. We show that the algorithm can be interpreted as an iterative latent semantic analysis process, which allows for extensions to handle networks with actor attributes and within-mode interactions. Experiments on both synthetic data and realworld networks demonstrate the efficacy of our approach and suggest its generality in capturing evolving groups in networks with heterogeneous entities and complex relationships.

View PDFchevron_right
From temporal network data to the dynamics of social relationships
Didier le bail

Proceedings of The Royal Society B: Biological Sciences, 2021

Networks are well-established representations of social systems, and temporal networks are widely used to study their dynamics. However, going from temporal network data, i.e., a stream of interactions between individuals, to a representation of the social group?s evolution, remains a challenge. Indeed, the temporal network at any specific time contains only the interactions taking place at that time and aggregating on successive time-windows also has important limitations. Here, we present a new framework to study the dynamic evolution of social networks based on the idea that social relationships are interdependent: as the time we can invest in social relationships is limited, reinforcing a relationship with someone is done at the expense of our relationships with others. We implement this interdependence in a parsimonious two-parameter model and apply it to several human and non-human primates? data sets to demonstrate that this model detects even small and short perturbations of the networks that cannot be detected using the standard technique of successive aggregated networks. Our model solves a long-standing problem by providing a simple and natural way to describe the dynamic evolution of social networks, with far-reaching consequences for the study of social networks and social evolution.

View PDFchevron_right
Fundamental limitations of network reconstruction from temporal data
Jaime A Moreno

Journal of the Royal Society, Interface, 2017

Inferring properties of the interaction matrix that characterizes how nodes in a networked system directly interact with each other is a well-known network reconstruction problem. Despite a decade of extensive studies, network reconstruction remains an outstanding challenge. The fundamental limitations governing which properties of the interaction matrix (e.g. adjacency pattern, sign pattern or degree sequence) can be inferred from given temporal data of individual nodes remain unknown. Here, we rigorously derive the necessary conditions to reconstruct any property of the interaction matrix. Counterintuitively, we find that reconstructing any property of the interaction matrix is generically as difficult as reconstructing the interaction matrix itself, requiring equally informative temporal data. Revealing these fundamental limitations sheds light on the design of better network reconstruction algorithms that offer practical improvements over existing methods.

View PDFchevron_right

References (34)

  1. Arbeitman, M., Furlong, E., Imam, F., Johnson, E., Null, B., Baker, B., Kras- now, M., Scott, M., Davis, R. and White, K. (2002). Gene expression during the life cycle of Drosophila melanogaster. Science 297 2270-2275.
  2. Banerjee, O., El Ghaoui, L. and d'Aspremont, A. (2008). Model selection through sparse maximum likelihood estimation. J. Mach. Learn. Res. 9 485-516. MR2417243
  3. Bresler, G., Mossel, E. and Sly, A. (2008). Reconstruction of Markov random fields from samples: Some observations and algorithms. In APPROX '08 / RANDOM '08: Proceedings of the 11th International Workshop, APPROX 2008, and 12th International Workshop, RANDOM 2008 on Approximation, Randomization and Combinatorial Op- timization 343-356. Springer, Berlin.
  4. Davidson, E. H. (2001). Genomic Regulatory Systems. Academic Press, San Diego.
  5. Drton, M. and Perlman, M. D. (2004). Model selection for Gaussian concentration graphs. Biometrika 91 591-602. MR2090624
  6. Duchi, J., Gould, S. and Koller, D. (2008). Projected subgradient methods for learning sparse Gaussians. In Proceedings of the Twenty-fourth Conference on Uncertainty in AI (UAI) 145-152.
  7. Efron, B., Hastie, T., Johnstone, I. and Tibshirani, R. (2004). Least angle regression. Ann. Statist. 32 407-499. MR2060166
  8. Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348-1360. MR1946581
  9. Fan J., Feng Y. and Wu, Y. (2009). Network exploration via the adaptive LASSO and SCAD penalties. Ann. Appl. Statist. 3 521-541.
  10. Friedman, J., Hastie, J. and Tibshirani, R. (2007). Sparse inverse co- variance estimation with the graphical lasso. Biostat, kxm045. Available at http://biostatistics.oxfordjournals.org/cgi/content/abstract/kxm045v1.
  11. Friedman, J., Hastie, T. and Tibshirani, R. (2008). Regularization paths for generalized linear models via coordinate descent. Technical report, Dept. Statistics, Stanford Univ.
  12. Friedman, J., Hastie, T., Hofling, H. and Tibshirani, R. (2007). Pathwise coordinate optimization. Ann. Appl. Statist. 1 302. MR2415737
  13. Getoor, L. and Taskar, B. (2007). Introduction to Statistical Relational Learning (Adap- tive Computation and Machine Learning). MIT Press, Cambridge, MA. MR2391486
  14. Grant, M. and Boyd, S. (2008). Cvx: Matlab software for disciplined convex program- ming (web page and software). Available at http://stanford.edu/ ~boyd/cvx.
  15. Guo, F., Hanneke, S., Fu, W. and Xing, E. P. (2007). Recovering temporally rewiring networks: A model-based approach. In Proceedings of the 24th International Conference on Machine Learning 321-328. ACM Press, New York.
  16. Hanneke, S. and Xing E. P. (2006). Discrete temporal models of social networks. Lecture Notes in Computer Science 4503 115-125.
  17. Koh, K., Kim, S.-J. and Boyd, S. (2007). An interior-point method for large-scale l1- regularized logistic regression. J. Mach. Learn. Res. 8 1519-1555. MR2332440
  18. Kolar, R. and Xing, E. P. (2009). Sparsistent estimation of time-varying discrete Markov random fields. ArXiv e-prints.
  19. Lauritzen, S. L. (1996). Graphical Models. Oxford Univ. Press, Oxford. MR1419991
  20. Luscombe, N., Babu, M., Yu, H., Snyder, M., Teichmann, S. and Gerstein, M. (2004). Genomic analysis of regulatory network dynamics reveals large topological changes. Nature 431 308-312.
  21. Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable se- lection with the lasso. Ann. Statist. 34 1436. MR2278363
  22. Peng, J., Wang, P., Zhou, N. and Zhu, J. (2009). Partial correlation estimation by joint sparse regression models. J. Amer. Statist. Assoc. 104 735-746. MR2541591
  23. Ravikumar, P., Wainwright, M. J. and Lafferty, J. D. (2010). High-dimensional ising model selection using ℓ1 regularized logistic regression. Ann. Statist. 38 1287-1319.
  24. Rinaldo, A. (2009). Properties and refinements of the fused lasso. Ann. Statist. 37 2922- 2952. MR2541451
  25. Rothman, A. J., Bickel, P. J., Levina, E. and Zhu, J. (2008). Sparse permutation invariant covariance estimation. Electron. J. Statist. 2 494. MR2417391
  26. Sarkar, P. and Moore, A. (2006). Dynamic social network analysis using latent space models. SIGKDD Explor. Newsl. 7 31-40.
  27. Tibshirani, R., Saunders, M., Rosset, S., Zhu, J. and Knight, K. (2005). Sparsity and smoothness via the fused lasso. J. Roy. Statist. Soc. Ser. B 67 91-108. MR2136641
  28. Tseng, P. (2001). Convergence of a block coordinate descent method for nondifferentiable minimization. J. Optim. Theory Appl. 109 475-494. MR1835069
  29. van Duijn, M. A. J., Gile, K. J. and Handcock, M. S. (2009). A framework for the comparison of maximum pseudo-likelihood and maximum likelihood estimation of ex- ponential family random graph models. Social Networks 31 52-62.
  30. Wainwright, M. J. and Jordan, M. I. (2008). Graphical models, exponential families, and variational inference. Found. Trends Mach. Learn. 1 1-305.
  31. Wainwright, M. J. (2009). Sharp thresholds for high-dimensional and noisy sparsity recovery using ℓ1-constrained quadratic programming (lasso). IEEE Trans. Inform. Theory 55 2183-2202.
  32. Watts, D. and Strogatz, S. (1998). Collective dynamics of 'small-world' networks. Na- ture 393 440-442.
  33. Yuan, M. and Lin, Y. (2007). Model selection and estimation in the Gaussian graphical model. Biometrika 94 19-35. MR2367824
  34. Zhou, S., Lafferty, J. and Wasserman, L. (2008). Time varying undirected graphs. In Conference on Learning Theory (R. A. Servedio and T. Zhang, eds.) 455-466. Omni- press, Madison, WI.

Related papers

Recovering Temporally Rewiring Networks: A Model-Based Approach
Steve Hanneke

Proceedings of the 24th …, 2007

View PDFchevron_right
Latent Variable Time-varying Network Inference
Saverio Salzo

Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018

In many applications of finance, biology and sociology, complex systems involve entities interacting with each other. These processes have the peculiarity of evolving over time and of comprising latent factors, which influence the system without being explicitly measured. In this work we present latent variable time-varying graphical lasso (LTGL), a method for multivariate time-series graphical modelling that considers the influence of hidden or unmeasurable factors. The estimation of the contribution of the latent factors is embedded in the model which produces both sparse and low-rank components for each time point. In particular, the first component represents the connectivity structure of observable variables of the system, while the second represents the influence of hidden factors, assumed to be few with respect to the observed variables. Our model includes temporal consistency on both components, providing an accurate evolutionary pattern of the system. We derive a tractable optimisation algorithm based on alternating direction method of multipliers, and develop a scalable and efficient implementation which exploits proximity operators in closed form. LTGL is extensively validated on synthetic data, achieving optimal performance in terms of accuracy, structure learning and scalability with respect to ground truth and state-of-the-art methods for graphical inference. We conclude with the application of LTGL to real case studies, from biology and finance, to illustrate how our method can be successfully employed to gain insights on multivariate time-series data.

View PDFchevron_right
Reconstructing complex networks from time series
Zoran Levnajić

2012

We propose a conceptually novel method of reconstructing the topology of dynamical networks. By examining the correlation between the variable of one node and the derivative of another node, we derive a simple matrix equation yielding the network adjacency matrix. Our assumptions are the possession of time series describing the network dynamics, and the precise knowledge of the interaction functions. Our method involves a tunable parameter, allowing for the reconstruction precision to be optimized within the constraints of given dynamical data. The method is illustrated on a simple example, and the dependence of the reconstruction precision on the dynamical properties of time series is discussed. Our theory is in principle applicable to any weighted or directed network whose internal interaction functions are known.

View PDFchevron_right
Network Inference from Perturbation Time Course Data
Caitlin Slay

2018

Networks underlie much of biology from subcellular to ecological scales. Yet, understanding what experimental data are needed and how to use them for unambiguously identifying the structure of even small networks remains a broad challenge. Here, we integrate a dynamic least squares framework into established modular response analysis (DL-MRA), that specifies sufficient experimental perturbation time course data to robustly infer arbitrary two and three node networks. DL-MRA considers important network properties that current methods often struggle to capture: (i) edge sign and directionality; (ii) cycles with feedback or feedforward loops including self-regulation; (iii) dynamic network behavior; (iv) edges external to the network; and (v) robust performance with experimental noise. We evaluate the performance of and the extent to which the approach applies to cell state transition networks, intracellular signaling networks, and gene regulatory networks. Although signaling networks ...

View PDFchevron_right
Inferring Time-Varying Network Topologies from Gene Expression Data
David States

EURASIP Journal on Bioinformatics and Systems Biology, 2007

Most current methods for gene regulatory network identification lead to the inference of steady-state networks, that is, networks prevalent over all times, a hypothesis which has been challenged. There has been a need to infer and represent networks in a dynamic, that is, time-varying fashion, in order to account for different cellular states affecting the interactions amongst genes. In this work, we present an approach, regime-SSM, to understand gene regulatory networks within such a dynamic setting. The approach uses a clustering method based on these underlying dynamics, followed by system identification using a state-space model for each learnt cluster-to infer a network adjacency matrix. We finally indicate our results on the mouse embryonic kidney dataset as well as the T-cell activation-based expression dataset and demonstrate conformity with reported experimental evidence.

View PDFchevron_right

Related topics

  • Statistics
  • Machine Learning
  • Convex Optimization
  • Statistical machine learning
  • Kernel Smoothing
  • Stochastic Networks
  • Regulatory Network
  • Structure learning
  • Graphical Model
  • Total Variation Regularization

    • Cited by

    The joint graphical lasso for inverse covariance estimation across multiple classes
    Patrick Danaher

    Journal of the Royal Statistical Society: Series B (Statistical Methodology), 2014

    We consider the problem of estimating multiple related but distinct graphical models on the basis of a high-dimensional data set with observations that belong to distinct classes. A motivating example occurs in the analysis of gene expression data for tissue samples with and without cancer. In this case, we might wish to estimate a gene expression network for the normal tissue and a gene expression network for the tumor tissue. We expect the two gene expression networks to be similar but not identical to each other, and so more accurate estimation of these two networks may be possible if they are estimated in a joint fashion. We propose the joint graphical lasso for this purpose. Rather than estimating a graphical model for each class separately, or estimating a single graphical model across all classes, we borrow strength across the classes in order to estimate multiple graphical models that share certain characteristics, such as the locations or weights of nonzero edges. Our approach is based upon maximizing a penalized log likelihood. We employ fused lasso or group lasso penalties, and implement a very fast computational approach that solves the joint graphical lasso problem exactly. In a simulation study we demonstrate that our proposed approach leads to more accurate estimation of networks and covariance structure than competing approaches. We further illustrate our proposal on a publicly-available lung cancer gene expression data set.

    View PDFchevron_right
    Bayesian graphical models for modern biological applications
    Francesco Stingo

    Statistical Methods & Applications

    Graphical models are powerful tools that are regularly used to investigate complex dependence structures in high-throughput biomedical datasets. They allow for holistic, systems-level view of the various biological processes, for intuitive and rigorous understanding and interpretations. In the context of large networks, Bayesian approaches are particularly suitable because it encourages sparsity of the graphs, incorporate prior information, and most importantly account for uncertainty in the graph structure. These features are particularly important in applications with limited sample size, including genomics and imaging studies. In this paper, we review several recently developed techniques for the analysis of large networks under non-standard settings, including but not limited to, multiple graphs for data observed from multiple related subgroups, graphical regression approaches used for the analysis of networks that change with covariates, and other complex sampling and structura...

    View PDFchevron_right
    Single timepoint models of dynamic systems
    Karen Sachs

    Interface focus, 2013

    Many interesting studies aimed at elucidating the connectivity structure of biomolecular pathways make use of abundance measurements, and employ statistical and information theoretic approaches to assess connectivities. These studies often do not address the effects of the dynamics of the underlying biological system, yet dynamics give rise to impactful issues such as timepoint selection and its effect on structure recovery. In this work, we study conditions for reliable retrieval of the connectivity structure of a dynamic system, and the impact of dynamics on structure-learning efforts. We encounter an unexpected problem not previously described in elucidating connectivity structure from dynamic systems, show how this confounds structure learning of the system and discuss possible approaches to overcome the confounding effect. Finally, we test our hypotheses on an accurate dynamic model of the IGF signalling pathway. We use two structure-learning methods at four time points to cont...

    View PDFchevron_right
    Nonparametric statistics of dynamic networks with distinguishable nodes
    Ricardo Fraiman

    TEST, 2017

    The study of random graphs and networks had an explosive development in the last couple of decades. Meanwhile, techniques for the statistical analysis of sequences of networks were less developed. In this paper we focus on networks sequences with a fixed number of labeled nodes and study some statistical problems in a nonparametric framework. We introduce natural notions of center and a depth function for networks that evolve in time. We develop several statistical techniques including testing, supervised and unsupervised classification, and some notions of principal component sets in the space of networks. Some examples and asymptotic results are given, as well as two real data examples.

    View PDFchevron_right
    Modeling of Information Processes in Social Networks
    Mikhail Kuptsov

    Information

    In order to model information dissemination in social networks, a special methodology of sampling statistical data formation has been implemented. The probability distribution laws of various characteristics of personal and group accounts in four social networks are investigated. Stochastic aspects of interrelations between these indicators were analyzed. The classification of groups of social network users is proposed, and their characteristic features and main empirical regularities of mutual transitions are marked. Regression models of forecasting changes in the number of users of the selected groups have been obtained.

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved