Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Discussion
Future Work
Conclusion
References
All Topics
Computer Science
Theoretical Computer Science

Spontaneous Emergence of Computation in Network Cascades

Profile image of Sotiris MoschoyiannisSotiris Moschoyiannis

2022

visibility

…

description

11 pages

link

1 file

Abstract

Neuronal network computation and computation by avalanche supporting networks are of interest to the fields of physics, computer science (computation theory as well as statistical or machine learning) and neuroscience. Here we show that computation of complex Boolean functions arises spontaneously in threshold networks as a function of connectivity and antagonism (inhibition), computed by logic automata (motifs) in the form of computational cascades. We explain the emergent inverse relationship between the computational complexity of the motifs and their rankordering by function probabilities due to motifs, and its relationship to symmetry in function space. We also show that the optimal fraction of inhibition observed here supports results in computational neuroscience, relating to optimal information processing.

Related papers

Macroscopic complexity from an autonomous network of networks of theta neurons
Ernest Barreto

Frontiers in Computational Neuroscience, 2014

View PDFchevron_right
Functional optimization in complex excitable networks
Joaquin Marro

EPL (Europhysics Letters), 2008

We study the effect of varying wiring in excitable random networks in which connection weights change with activity to mold local resistance or facilitation due to fatigue. Dynamic attractors, corresponding to patterns of activity, are then easily destabilized according to three main modes, including one in which the activity shows chaotic hopping among the patterns. We describe phase transitions to this regime, and show a monotonous dependence of critical parameters on the heterogeneity of the wiring distribution. Such correlation between topology and functionality implies, in particular, that tasks which require unstable behavior -such as pattern recognition, family discrimination and categorization-can be most efficiently performed on highly heterogeneous networks. It also follows a possible explanation for the abundance in nature of scale-free network topologies.

View PDFchevron_right
1 Project Temporal Complexity in Biologically Plausible Formal Neural Networks
Brigitte Quenet
View PDFchevron_right
Complexity in Neuronal Networks
Alain Destexhe

Figure 1: Schematic levels of spatial integration in the nervous system. The spatial scale at which anatomical organizations can be identified varies over many orders of magnitude. Icons to the right represent structures at distinct levels in a bottom-up fashion: (bottom) a chemical synapse, (middle-bottom) a network model of how thalamic afferent cells could be connected to simple cells in visual cortex, (middle-top) maps of orientation preference and ocular dominance in a primary visual area; (top) the subset of visual areas forming visual cortex and their interconnections (adapted from ref. [1]).

View PDFchevron_right
Characterization of multiscale logic operations in the neural circuits
Kyungreem Han

Frontiers in Bioscience-Landmark

View PDFchevron_right
Activities and Sensitivities in Boolean Network Models
Stuart Kauffman

Physical Review Letters, 2004

We study how the notions of importance of variables in Boolean functions as well as the sensitivities of the functions to changes in these variables impact the dynamical behavior of Boolean networks. The activity of a variable captures its influence on the output of the function and is a measure of that variable's importance. The average sensitivity of a Boolean function captures the smoothness of the function and is related to its internal homogeneity. In a random Boolean network, we show that the expected average sensitivity determines the well-known critical transition curve. We also discuss canalizing functions and the fact that the canalizing variables enjoy higher importance, as measured by their activities, than the noncanalizing variables. Finally, we demonstrate the important role of the average sensitivity in determining the dynamical behavior of a Boolean network.

View PDFchevron_right
Binary threshold networks as a natural null model for biological networks
Stefan Bornholdt

Physical Review E, 2012

Spin models of neural networks and genetic networks are considered elegant as they are accessible to statistical mechanics tools for spin glasses and magnetic systems. However, the conventional choice of variables in spin systems may cause problems in some models when parameter choices are unrealistic from a biological perspective. Obviously, this may limit the role of a model as a template model for biological systems. Perhaps less obviously, also ensembles of random networks are affected and may exhibit different critical properties. We consider here a prototypical network model that is biologically plausible in its local mechanisms. We study a discrete dynamical network with two characteristic properties: Nodes with binary states 0 and 1, and a modified threshold function with Θ0(0) = 0. We explore the critical properties of random networks of such nodes and find a critical connectivity Kc = 2.0 with activity vanishing at the critical point. Finally, we observe that the present model allows a more natural implementation of recent models of budding yeast and fission yeast cell-cycle control networks.

View PDFchevron_right
Criticality and Computation in Random Threshold Networks with Noise
Ruedi Stoop

2008

Abstract—Boolean networks are considered generic models for a large class of asymmetric networks, such as neural networks or gene regulatory networks. Previous work has shown that such networks undergo a phase transition from an ordered to a'chaotic'phase as ...

View PDFchevron_right
The complexity of latching transitions in large scale cortical networks
Emilio Kropff, Alessandro Treves

Natural Computing, 2007

We study latching dynamics, i.e. the ability of a network to hop spontaneously from one discrete attractor state to another, which has been proposed as a model of an infinitely recursive process in large scale cortical networks, perhaps associated with higher cortical functions, such as language. We show that latching dynamics can span the range from deterministic to random under the control of a threshold parameter U. In particular, the interesting intermediate case is characterized by an asymmetric and complex set of transitions. We also indicate how finite latching sequences can become infinite, depending on the properties of the transition probability matrix and of its eigenvalues.

View PDFchevron_right
On Properties of Networks of Neuron-Like Elements
Pierre Baldi

Nips, 1987

The complexity and computational capacity of multi-layered, feedforward neural networks is examined. Neural networks for special purpose (structured) functions are examined from the perspective of circuit complexity. Known results in complexity theory are applied to the special instance of neural network circuits, and in particular, classes of functions that can be implemented in shallow circuits characterised. Some conclusions are also drawn about learning complexity, and some open problems raised. The dual problem of determining the computational capacity of a class of multi-layered networks with dynamics regulated by an algebraic Hamiltonian is considered. Formal results are presented on the storage capacities of programmed higher-order structures, and a tradeoff between ease of programming and capacity is shown. A precise determination is made of the static fixed point structure of random higher-order constructs, and phase-transitions (0-1 laws) are shown.

View PDFchevron_right

References (44)

  1. P. W. Anderson. More is different: broken symmetry and the nature of the hierarchical structure of science. Science, 177(4047):393-396, 1972.
  2. J. M. Beggs. The criticality hypothesis: how local cortical networks might optimize information process- ing. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 366(1864):329-343, 2008.
  3. A. Ben-Naim. A farewell to entropy: Statistical thermodynamics based on information: S. World Scientific, 2008.
  4. D. R. Brooks, E. O. Wiley, and D. Brooks. Evolution as entropy. University of Chicago Press Chicago, 1988.
  5. V. Capano, H. J. Herrmann, and L. De Arcangelis. Optimal percentage of inhibitory synapses in multi-task learning. Scientific reports, 5(1):1-5, 2015.
  6. K. Christensen and N. R. Moloney. Complexity and criticality, volume 1. World Scientific Publishing Company, 2005.
  7. J. Conway et al. The game of life. Scientific American, 223(4):4, 1970.
  8. J. M. Dubbey and J. M. Dubbey. The mathematical work of Charles Babbage. Cambridge University Press, 2004.
  9. D. Easley, J. Kleinberg, et al. Networks, crowds, and markets, volume 8. Cambridge university press Cambridge, 2010.
  10. M. Granovetter. Threshold models of collective behavior. American journal of sociology, 83(6):1420-1443, 1978.
  11. J. Hesse and T. Gross. Self-organized criticality as a fundamental property of neural systems. Frontiers in systems neuroscience, 8:166, 2014.
  12. M. Jalili and M. Perc. Information cascades in complex networks. Journal of Complex Networks, 5(5):665-693, 2017.
  13. E. T. Jaynes. Information theory and statistical mechanics. Physical review, 106(4):620, 1957.
  14. H. J. Jensen. What is critical about criticality: in praise of the correlation function. Journal of Physics: Complexity, 2(3):032002, 2021.
  15. D. Kempe, J. Kleinberg, and É. Tardos. Maximizing the spread of influence through a social network. In Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 137-146, 2003.
  16. K. H. Knuth. Information physics: The new frontier. In AIP Conference Proceedings, volume 1305, pages 3-19. American Institute of Physics, 2011.
  17. L. Landau. Broken symmetry and phase transitions. Phys Z Sowjetunion, 1937, 11, 26, 1937.
  18. R. Landauer. Irreversibility and heat generation in the computing process. IBM journal of research and development, 5(3):183-191, 1961.
  19. J. T. Lizier, M. Prokopenko, and A. Y. Zomaya. A framework for the local information dynamics of distributed computation in complex systems. In Guided self-organization: inception, pages 115-158. Springer, 2014.
  20. S. Lloyd. The computational universe. Information and the nature of reality: From physics to metaphysics, pages 92-103, 2010.
  21. S. Lloyd. The universe as quantum computer. A Computable Universe: Understanding and exploring Nature as computation, pages 567-581, 2013.
  22. C. W. Lynn and D. S. Bassett. The physics of brain network structure, function and control. Nature Reviews Physics, 1(5):318-332, 2019.
  23. P. Massobrio, L. de Arcangelis, V. Pasquale, H. J. Jensen, and D. Plenz. Criticality as a signature of healthy neural systems. Frontiers in systems neuroscience, 9:22, 2015.
  24. R. Milo, S. Shen-Orr, S. Itzkovitz, N. Kashtan, D. Chklovskii, and U. Alon. Network motifs: simple building blocks of complex networks. Science, 298(5594):824-827, 2002.
  25. M. Newman. Networks. Oxford university press, 2018.
  26. S. Perseguers, M. Lewenstein, A. Acín, and J. I. Cirac. Quantum random networks. Nature Physics, 6(7):539-543, 2010.
  27. R. Rojas. Neural networks: a systematic introduction. Springer Science & Business Media, 2013.
  28. J. M. Sakoda. The checkerboard model of social interaction. The Journal of Mathematical Sociology, 1(1):119-132, 1971.
  29. J. E. Savage. Models of computation, volume 136. Addison-Wesley Reading, MA, 1998.
  30. T. C. Schelling. Dynamic models of segregation. Journal of mathematical sociology, 1(2):143-186, 1971.
  31. R. Schrodinger, E. Schrödinger, and E. S. Dinger. What is life?: With mind and matter and autobiographical sketches. Cambridge university press, 1992.
  32. T. J. Sejnowski, C. Koch, and P. S. Churchland. Computational neuroscience. Science, 241(4871):1299-1306, 1988.
  33. C. E. Shannon. Mathematical theory of the differential analyzer. Journal of Mathematics and Physics, 20(1- 4):337-354, 1941.
  34. C. E. Shannon. A mathematical theory of communication. The Bell system technical journal, 27(3):379-423, 1948.
  35. W. L. Shew and D. Plenz. The functional benefits of criticality in the cortex. The neuroscientist, 19(1):88-100, 2013.
  36. G. Tononi, O. Sporns, and G. M. Edelman. A measure for brain complexity: relating functional segregation and integration in the nervous system. Proceedings of the National Academy of Sciences, 91(11):5033-5037, 1994.
  37. A. M. Turing et al. On computable numbers, with an application to the entscheidungsproblem. J. of Math, 58(345-363):5, 1936.
  38. J. Von Neumann. The general and logical theory of automata, cerebral mechanisms in behavior. the hixon symposium. New York: John Wiley&Sons, 1951.
  39. J. Von Neumann. Probabilistic logics and the synthesis of reliable organisms from unreliable components. Automata Studies, pages 43-98, 1956.
  40. D. Watts. A simple model of global cascades on random networks. Proceedings of the National Academy of Sciences of the United States of America, 99(9):5766-5771, 2002.
  41. J. A. Wheeler. Recent thinking about the nature of the physical world: It from bit a. Annals of the New York Academy of Sciences, 655(1):349-364, 1992.
  42. J. A. Wheeler. Information, physics, quantum: The search for links. CRC Press, 2018.
  43. G. Wilkerson and S. Moschoyiannis. Universal boolean logic in cascading networks. In International conference on complex networks and their applications, pages 601-611. Springer, 2019.
  44. S. Wolfram. A new kind of science, volume 5. Wolfram media Champaign, IL, 2002.

Related papers

On Computing Structural and Behavioral Complexities of Threshold Boolean Networks
Nicolas Glade

Acta Biotheoretica

Various threshold Boolean networks (TBNs), a formalism used to model different types of biological networks (genes notably), can produce similar dynamics, i.e. share same behaviors. Among them, some are complex (according to Kolmogorov complexity), others not. By computing both structural and behavioral complexities, we show that most TBNs are structurally complex, even those having simple behaviors. For this purpose, we developed a new method to compute the structural complexity of a TBN based on estimates of the sizes of equivalence classes of the threshold Boolean functions composing the TBN.

View PDFchevron_right
Logic and learning in network cascades
Sotiris Moschoyiannis

2021

Critical cascades are found in many self-organizing systems. Here, we examine critical cascades as a design paradigm for logic and learning under the linear threshold model (LTM), and simple biologically inspired variants of it as sources of computational power, learning efficiency, and robustness. First, we show that the LTM can compute logic, and with a small modification, universal Boolean logic, examining its stability and cascade frequency. We then frame it formally as a binary classifier and remark on implications for accuracy. Second, we examine the LTM as a statistical learning model, studying benefits of spatial constraints and criticality to efficiency. We also discuss implications for robustness in information encoding. Our experiments show that spatial constraints can greatly increase efficiency. Theoretical investigation and initial experimental results also indicate that criticality can result in a sudden increase in accuracy.

View PDFchevron_right
Robust motifs of threshold-linear networks
Christopher Langdon

arXiv: Neurons and Cognition, 2019

To any inhibition-dominated threshold-linear network (TLN) we can associate a directed graph that captures the pattern of strong and weak inhibition between neurons. Robust motifs are graphs for which the structure of fixed points in the network is independent of the choice of connectivity matrix $W$, and whose dynamics are thus greatly constrained. This makes them ideal building blocks for constructing larger networks whose behaviors are robust to changes in connection strengths. In contrast, flexible motifs correspond to networks with multiple dynamic regimes. In this work, we give a purely graphical characterization of both flexible and robust motifs of any size. We find that all but a few robust motifs fall into two infinite families of graphs with simple feedforward architectures.

View PDFchevron_right
A biologically constrained learning mechanism in networks of formal neurons
Gerard DREYFUS

Journal of Statistical Physics, 1986

A new learning mechanism is proposed for networks of formal neurons analogous to Ising spin systems; it brings such models substantially closer to biological data in three respects: first, the learning procedure is applied initially to a network with random connections (which may be similar to a spin-glass system), instead of starting from a system void of any knowledge (as in the Hopfield model); second, the resultant couplings are not symmetrical; third, patterns can be stored without changing the sign of the coupling coefficients. It is shown that the storage capacity of such networks is similar to that of the Hopfield network, and that it is not significantly affected by the restriction of keeping the couplings' signs constant throughout the learning phase. Although this approach does not claim to model the central nervous system, it provides new insight on a frontier area between statistical physics, artificial intelligence, and neurobiology.

View PDFchevron_right
Boolean Threshold Networks: Virtues and Limitations for Biological Modeling
Jorge G. T. Zañudo, Gustavo Martínez-mekler

Boolean threshold networks have recently been proposed as useful tools to model the dynamics of genetic regulatory networks, and have been successfully applied to describe the cell cycles of S. cerevisiae and S. pombe. Threshold networks assume that gene regulation processes are additive. This, however, contrasts with the mechanism proposed by S. Kauffman in which each of the logic functions must be carefully constructed to accurately take into account the combinatorial nature of gene regulation. While Kauffman Boolean networks have been extensively studied and proved to have the necessary properties required for modeling the fundamental characteristics of genetic regulatory networks, not much is known about the essential properties of threshold networks. Here we study the dynamical properties of these networks with different connectivities, activator-repressor proportions, activator-repressor strengths and different thresholds. Special attention is paid to the way in which the threshold value affects the dynamical regime in which the network operates and the structure of the attractor landscape. We find that only for a very restricted set of parameters, these networks show dynamical properties consistent with what is observed in biological systems. The virtues of these properties and the possible problems related with the restrictions are discussed and related to earlier work that uses these kind of models.

View PDFchevron_right

Related topics

  • Computer Science
  • Physics
  • Theoretical Computer Science
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved