Motivated by the possible applications that a better understanding of consciousness might bring, ... more Motivated by the possible applications that a better understanding of consciousness might bring, we follow Tononi's idea and calculate analytically a complexity index for two systems of Ising spins with parallel update dynamics, the homogeneous and a modular infinite range models. Using the information geometry formulation of integrated information theory, we calculate the geometric integrated information index, φ G (Π) for a fixed partition Π with K components and Φ =max Π φ G (Π) for K = 2 or 3. For systems in the deep ferromagnetic phase, the optimal partition undergoes a transition such that the smallest (largest) component is above (resp. below) its critical temperature. The effects of partitioning are taken into account by introducing site dilution.
We study the interaction of agents, where each one consists of an associative memory neural netwo... more We study the interaction of agents, where each one consists of an associative memory neural network trained with the same memory patterns and possibly different reinforcement-unlearning dreaming periods. Using replica methods, we obtain the rich equilibrium phase diagram of the coupled agents. It shows phases such as the student-professor phase, where only one network benefits from the interaction while the other is unaffected; a mutualism phase, where both benefit; an indifferent phase and an insufficient phase, where neither are benefited nor impaired; a phase of amensalism where one is unchanged and the other is damaged. In addition to the paramagnetic and spin glass phases, there is also one we call the reinforced delusion phase, where agents concur without having finite overlaps with memory patterns. For zero coupling constant, the model becomes the reinforcement and removal dreaming model, which without dreaming is the Hopfield model. For finite coupling and a single memory pattern, it becomes a Mattis version of the Ashkin-Teller model.
We introduce and study an artificial neural network, inspired by the probabilistic Receptor Affin... more We introduce and study an artificial neural network, inspired by the probabilistic Receptor Affinity Distribution model of olfaction. Our system consists on N sensory neurons whose outputs converge on a single processing linear threshold element. The system's aim is to model discrimination of a single target odorant from a large number p of background odorants, within a range of odorant concentrations. We show that this is possible provided p does not exceed a critical value pc, and calculate the critical capacity αc = pc/N. The critical capacity depends on the range of concentrations in which the discrimination is to be accomplished. If the olfactory bulb may be thought of as a collection of such processing elements, each responsible for the discrimination of a single odorant, our study provides a quantitative analysis of the potential computational properties of the olfactory bulb. The mathematical formulation of the problem we consider is one of determining the capacity for linear separability of continuous curves, embedded in a large dimensional space. This is accomplished here by a numerical study, using a method that signals whether the discrimination task is realizable or not, together with a finite size scaling analysis.
A neural network with a learning algorithm optimized by information theory entropic dynamics is u... more A neural network with a learning algorithm optimized by information theory entropic dynamics is used to build an agent dubbed Homo Entropicus. The algorithm can be described at a macroscopic level in terms of aggregate variables interpretable as quantitative markers of proto-emotions. We use systems of such interacting neural networks to construct a framework for modeling societies that show complex emergent behavior. A few applications are presented to investigate the role the interactions of opinions about multidimensional issues and trust on the information source play on the state of the agent society. These include the case of a class of N agents learning from a fixed teacher; two dynamical agents; panels of three agents modeling the interactions that occur in decisions of the US Court of Appeals, where we quantify how politically biased are the agents, how trustful of other agents-judges of other parties, how much the agents follow a common understanding of the law. Finally we address under which conditions ideological polarization follows or precedes affective polarization in large societies and how simpler versions of the learning algorithm may change these relations.
We propose and analyze two different Bayesian online algorithms for learning in discrete Hidden M... more We propose and analyze two different Bayesian online algorithms for learning in discrete Hidden Markov Models and compare their performance with the already known Baldi-Chauvin Algorithm. Using the Kullback-Leibler divergence as a measure of generalization we draw learning curves in simplified situations for these algorithms and compare their performances.
Trust, law and ideology in a NN agent model of the US Appellate Courts
Interacting NN are used to model US Appellate Court three judge panels. Agents, whose initial sta... more Interacting NN are used to model US Appellate Court three judge panels. Agents, whose initial states have three contributions derived from common knowledge of the law, political affiliation and personality, learn by exchange of opinions, updating their state and trust about other agents. The model replicates data patterns only if initially the agents trust each other and are certain about their trust independently of party affiliation, showing evidence of ideological voting, dampening and amplification. Absence of law or party contribution destroys the theoretical-empirical agreement. We identify quantitative signatures for different levels of the law, ideological or idiosyncratic contributions.
We employ the Bayesian framework to define a cointegration distance aimed to represent long term ... more We employ the Bayesian framework to define a cointegration distance aimed to represent long term relationships between time series. For visualization and clustering of these relationships we calculate a distance matrix and introduce a map based on the Sorting Points Into Neighborhoods (SPIN) technique, which has been previously used to analyze large data sets from DNA arrays. We exemplify the technique in three data sets: US interest rates, monthly inflation rates and gross domestic product growth rates.
We study maximum entropy mechanisms of information exchange between agents modeled by neural netw... more We study maximum entropy mechanisms of information exchange between agents modeled by neural networks and the macroscopic states of a society of such agents in a few situations. Mathe10 matical quantification of surprise, distrust of other agents and confidence about its opinion emerge as essential ingredients in the entropy based learning dynamics. Learning is shown to be driven by surprises, i.e. the receptor agent is confronted with the concurring opinion of a distrusted agent or with a trusted agent’s disagreeing opinion. Attribution of blame for the surprise derives from measures of distrust of the receiver towards the emitter agent and the receiver’s confidence about its 15 own opinion. The dynamics proceeds by changes of mainly one or the other: the receptor opinion about the issue or the distrust about the emitter. A society with N agents exchanging binary opinions about a set of issues show rich behavior which depend on the complexity of the agenda. For small sets the socie...
Mean Field Studies of a Society of Interacting Agents
We model a society of agents that interact in pairs by exchanging for/against opinions about issu... more We model a society of agents that interact in pairs by exchanging for/against opinions about issues using an algorithm obtained with methods of Bayesian inference and maximum entropy. The agents gauge the incoming information with respect to the mistrust attributed to the other agents. There is no underlying lattice and all agents interact among themselves. The interaction pair can be described as a dynamics along the gradient of the logarithm of the evidence. By using a symmetric version of the two-body interactions we introduce a Hamiltonian for the whole society. Knowledge of the expected value of the Hamiltonian is relevant information for the state of the society. In the case of uniform mistrust, independent of the pair of agents, the phase diagram of the society in a mean field approximation shows a phase transition that separates an ordered phase where opinions are to a large extent shared by the agents and a disordered phase of dissension of opinions.
Empirical evidence suggests that social structure may have changed from hierarchical to egalitari... more Empirical evidence suggests that social structure may have changed from hierarchical to egalitarian and back along the evolutionary line of humans. We model a society subject to competing cognitive and social navigation constraints. The theory predicts that the degree of hierarchy decreases with encephalization and increases with group size. Hence hominin groups may have been driven from a phase with hierarchical order to a phase with egalitarian structures by the encephalization during the last two million years, and back to hierarchical due to fast demographical changes during the Neolithic. The dynamics in the perceived social network shows evidence in the egalitarian phase of the observed phenomenon of Reverse Dominance. The theory also predicts for modern hunter-gatherers in mild climates a trend towards an intermediate hierarchy degree and a phase transition for harder ecological conditions. In harsher climates societies would tend to bemore egalitarian if organized in small g...
Predição de classes de enzimas usando método de agrupamento super-paramagnético
As proteinas com atividade enzimatica podem ser divididas em seis grandes classes de acordo com s... more As proteinas com atividade enzimatica podem ser divididas em seis grandes classes de acordo com suas funcoes especificas, que sao: oxirredutases, hidrolases, transferases, lyases, isomerases e ligases. Utilizando algoritmo de agrupamento nao-parametrico, este trabalho tem como objetivo predizer diferentes classes enzimaticas usando parâmetros estruturais e fisico-quimicos.
Bayesian algorithms pose a limit to the performance learning algorithms can achieve. Natural sele... more Bayesian algorithms pose a limit to the performance learning algorithms can achieve. Natural selection should guide the evolution of information processing systems towards those limits. What can we learn from this evolution and what properties do the intermediate stages have? While this question is too general to permit any answer, progress can be made by restricting the class of information processing systems under study. We present analytical and numerical results for the evolution of on-line algorithms for learning from examples for neural network classifiers, which might include or not a hidden layer. The analytical results are obtained by solving a variational problem to determine the learning algorithm that leads to maximum generalization ability. Simulations using evolutionary programming, for programs that implement learning algorithms, confirm and expand the results. The principal result is not just that the evolution is towards a Bayesian limit. Indeed it is essentially reached. In addition we find that evolution is driven by the discovery of useful structures or combinations of variables and operators. In different runs the temporal order of the discovery of such combinations is unique. The main result is that combinations that signal the surprise brought by an example arise always before combinations that serve to gauge the performance of the learning algorithm. This latter structures can be used to implement annealing schedules. The temporal ordering can be understood analytically as well by doing the functional optimization in restricted functional spaces. We also show that there is data suggesting that the appearance of these traits also follows the same temporal ordering in biological systems.
The exchange of ideas between statistical physics and computer science has been very fruitful and... more The exchange of ideas between statistical physics and computer science has been very fruitful and is currently gaining momentum as a consequence of the revived interest in neural networks, machine learning and inference in general. Statistical physics methods complement other approaches to the theoretical understanding of machine learning processes and inference in stochastic modeling. They facilitate, for instance, the study of dynamical and equilibrium properties of randomized training processes in model situations. At the same time, the approach inspires novel and efficient algorithms and facilitates interdisciplinary applications in a variety of scientific and technical disciplines.
Resumo Anedotas sobre Feynman são abundantes e cada um que teve contato com ele, das mais diversa... more Resumo Anedotas sobre Feynman são abundantes e cada um que teve contato com ele, das mais diversas formas, tem algo a contar. Este texto não inclui nenhuma física e nem o propósito de discutir algo técnico. Simplesmente traz algumas lembranças das aulas de física teórica dadas por Richard P. Feynman e do ambiente no California Institute of Technology no começo da década de oitenta. Mais do que técnicas, a mensagem transmitida era sobre o prazer de fazer ciência, sobre a coragem frente à dúvida e sobre a honestidade intelectual. As citações são traduações do original na minha memória.
O objetivo deste artigo é mostrar em casos simples como funcionam as redes neurais. Nesse sentido... more O objetivo deste artigo é mostrar em casos simples como funcionam as redes neurais. Nesse sentido, embora seja possível descrever o funcionamento de uma rede neural de arquitetura profunda de várias formas, neste artigo optou-se pela descrição em termos da construção e reconstrução de representações internas à medida que a informação se propaga pela rede.
While social interactions tend to decrease differences in opinions, multiplicity of groups and in... more While social interactions tend to decrease differences in opinions, multiplicity of groups and individual opinion differences persist in human societies. Axelrod identified homophily and social conformity seeking as basic interactions that can lead to multiculturalism in spatial scenarios in models under certain special conditions. We follow another route, where the social interaction between any two agents is given by the descent along the gradient of a cost function deduced from a Bayesian learning formalism. The cost function depends on a hyperparameter that estimates the trust of one agent in the information provided by the other. If the expected value of the total cost function is relevant information, Maximum Entropy permits characterizing the state of the society. Furthermore we introduce a dynamics on the trust parameters, which increases when agents concur and decreases otherwise. We study the resulting phase diagram in the case of large number of interacting agents on a complete social graph, hence under sympatric conditions. Simulations show that there is evolution of assortative distrust in rich cultural environments measured by the diversity of the set of issues under discussions. High distrust leads to antilearning which leads to multiple groups which hold different opinions on the set of issues. We simulate conditions of political pressure and interaction that describe the House of Congress of Brazil and are able to qualitatively replicate voting patterns through four presidential cycles during the years of 1994 to 2010.
Uploads
Papers by Nestor Caticha