Academia.eduAcademia.edu

Outline

Title
Abstract
Figures
Completeness Results
References
All Topics
Philosophy
Logic

How Logic Emerges from the Dynamics of Information

Profile image of Peter GardenforsPeter Gardenfors

1994

Abstract

It is often claimed that the symbolic approach to information processing is incompatible with connectionism and other associationist modes of representing information. I propose to throw new light on this debate by presenting two examples of how logic can be seen as emerging from an underlying information dynamics. The first example shows how intuitionistic logic results very naturally from an abstract analysis of the dynamics of information. The second example establishes that the activities of a large class of neural networks may be interpreted, on the symbolic level, as nonmonotonic inferences. On the basis of these examples I argue that symbolic and non-symbolic approaches to information can be described in terms of different perspectives on the same phenomenon. Thus, I find that Fodor and Pylyshyn's claim that connectionist systems cannot be systematic and compositional is based on a misleading interpretation of representations in such systems.

Figures (4)
Example: The network depicted below consists of four neurons with activities x},....x4. Neurons that interact are connected by lines. Arrows at the ends of the lines indicate that the neurons excite each other; dots indicate that they inhibit each other. If we consider only schemata corresponding to binary activity vectors, it is possible to identify schemata with sets of active neurons. Let three schemata a, B, and y correspond to the following activity vectors: a=<1 1 0 0>, B=<0 0 0 1>, y=<0 1 1 0O>. Assume that x4 inhibits x3 more than x2 excites x3. If a is given as input, the network will activate x3 and thus y. It follows that [a] > y and hence a 1 y. Extending the input to a*B causes the network to withdraw y, 1.e., no longer represent this schema, since the activity in x4 inhibits x3. In formal terms as , y. One way of characterizing the nonmonotonic inferences generated by a neural network is to study them in terms of general postulates for nonmonotonic logics that have been investigated in the literature (Gabbay 1985, Makinson 1993, Kraus, Lehmann and Magidor 1991, Makinson and Gé§rdenfors 1991, Gardenfors and Makinson 1993).
Example: The network depicted below consists of four neurons with activities x},....x4. Neurons that interact are connected by lines. Arrows at the ends of the lines indicate that the neurons excite each other; dots indicate that they inhibit each other. If we consider only schemata corresponding to binary activity vectors, it is possible to identify schemata with sets of active neurons. Let three schemata a, B, and y correspond to the following activity vectors: a=<1 1 0 0>, B=<0 0 0 1>, y=<0 1 1 0O>. Assume that x4 inhibits x3 more than x2 excites x3. If a is given as input, the network will activate x3 and thus y. It follows that [a] > y and hence a 1 y. Extending the input to a*B causes the network to withdraw y, 1.e., no longer represent this schema, since the activity in x4 inhibits x3. In formal terms as , y. One way of characterizing the nonmonotonic inferences generated by a neural network is to study them in terms of general postulates for nonmonotonic logics that have been investigated in the literature (Gabbay 1985, Makinson 1993, Kraus, Lehmann and Magidor 1991, Makinson and Gé§rdenfors 1991, Gardenfors and Makinson 1993).
A critical factor here seems to be the J/inear summation of inputs that ’locks’ x2 and x3 to inputs from the outside because the inhibitory connection between them is large.
A critical factor here seems to be the J/inear summation of inputs that ’locks’ x2 and x3 to inputs from the outside because the inhibitory connection between them is large.
Counterexample to Or: The following network is a simple counterexample: xj excites x4 more than x2 inhibits x4. The same is true for x3 and x2. Giving a=<1100> or B=<01 10> as input activates x4, thus a | y and B Uy. On the other hand, the neuron x2 which represents schema aH has only inhibitory connections to x4. As a consequence aH , y. In summary, following Balkenius and G§rdenfors (1991), it has been shown that by introducing an appropriate schema concept and exploiting the higher- level features of a resonance function in a neural network it is possible to define a form of nonmonotonic inference relation. It has also been established that this inference relation satisfies some of the most fundamental postulates for nonmonotonic logics.
Counterexample to Or: The following network is a simple counterexample: xj excites x4 more than x2 inhibits x4. The same is true for x3 and x2. Giving a=<1100> or B=<01 10> as input activates x4, thus a | y and B Uy. On the other hand, the neuron x2 which represents schema aH has only inhibitory connections to x4. As a consequence aH , y. In summary, following Balkenius and G§rdenfors (1991), it has been shown that by introducing an appropriate schema concept and exploiting the higher- level features of a resonance function in a neural network it is possible to define a form of nonmonotonic inference relation. It has also been established that this inference relation satisfies some of the most fundamental postulates for nonmonotonic logics.
In this network “[d]rawing an inference from A&B to A thus corresponds to an excitation of node 2 being caused by an excitation of node 1” (p. 15).2° The fundamental mistake in this example is that disjoint nodes are assumed to represent different expres- sions.”! This assumption conflates representations on the symbolic level (which is where representations of expressions and inferences belong) with represen- tations on the connectionist level (where representa- tions of associations are handled). Smolensky (1991a, p. 206) argues concerning Fodor and Pylyshyn’s example that
In this network “[d]rawing an inference from A&B to A thus corresponds to an excitation of node 2 being caused by an excitation of node 1” (p. 15).2° The fundamental mistake in this example is that disjoint nodes are assumed to represent different expres- sions.”! This assumption conflates representations on the symbolic level (which is where representations of expressions and inferences belong) with represen- tations on the connectionist level (where representa- tions of associations are handled). Smolensky (1991a, p. 206) argues concerning Fodor and Pylyshyn’s example that

Related papers

A Defence Of Connectionism Against The" Syntactic''Argument
Marcello Guarini

Synthese, 2001

ABSTRACT. In “Representations without Rules, Connectionism and the Syntactic Argument”, Kenneth Aizawa argues against the view that connectionist nets can be un-derstood as processing representations without the use of representation-level rules, and he ...

View PDFchevron_right
Representational systems and symbolic systems
Mike Oaksford

Behavioral and Brain Sciences, 1990

Connectionist models provide a promising alternative to the traditional computational approach that has for several decades dominated cognitive science and artificial intelligence, although the nature of connectionist models and their relation to symbol processing remains controversial. Connectionist models can be characterized by three general computational features: distinct layers of interconnected units, recursive rules for updating the strengths of the connections during learning, and "simple" homogeneous computing elements. Using just these three features one can construct surprisingly elegant and powerful models of memory, perception, motor control, categorization, and reasoning. What makes the connectionist approach unique is not its variety of representational possibilities (including "distributed representations") or its departure from explicit rule-based models, or even its preoccupation with the brain metaphor. Rather, it is that connectionist models can be used to explore systematically the complex interaction between learning and representation, as we try to demonstrate through the analysis of several large networks.

View PDFchevron_right
Logic in Cognitive Science: Bridging the Gap between Symbolic and Connectionist Paradigms
Jakub Szymanik

2010

What can logic contribute to cognitive science? In the early days of cognitive science, logic was taken to play both a descriptive and a normative role in theories of intelligent behavior. Descriptively, human beings were taken to be fundamentally logical, or rational. Normatively, logic was taken to define rational behavior and thus to provide a starting point for the artificial reproduction of intelligence. Both positions were soon challenged.

View PDFchevron_right
Reasoning, nonmonotonicity and learning in connectionist networks that capture propositional knowledge
Gadi Pinkas

Artificial Intelligence, 1995

The paper presents a connectionist framework that is capable of representing and learning propositional knowledge. An extended version of propositional calculus is developed and is demonstrated to be useful for nonmonotonic reasoning, dealing with conflicting beliefs and for coping with inconsistency generated by unreliable knowledge sources. Formulas of the extended calculus are proved to be equivalent in a very strong sense to symmetric networks (like Hopfield networks and Boltzmann machines), and efficient algorithms are given for translating back and forth between the two forms of knowledge representation. A fast learning procedure is presented that allows symmetric networks to learn representations of unknown logic formulas by looking at examples. A connectionist inference engine is then sketched whose knowledge is either compiled from a symbolic representation or learned inductively from training examples. Experiments with large scale randomly generated formulas suggest that the parallel local search that is executed by the networks is extremely fast on average. Finally, it is shown that the extended logic can be used as a high-level specification language for connectionist networks, into which several recent symbolic systems may be mapped. The paper demonstrates how a rigorous bridge can be constructed that ties together the (sometimes opposing) connectionist and symbolic approaches.

View PDFchevron_right
Special issue on integration of symbolic and connectionist systems
Franz Kurfess

Cognitive Systems Research, 2002

View PDFchevron_right
Nonmonotonic Inferences and Neural Networks
Reinhard Blutner

Synthese, 2004

There is a gap between two different modes of computation: the symbolic mode and the subsymbolic (neuron-like) mode. The aim of this paper is to overcome this gap by viewing symbolism as a high-level description of the properties of (a class of) neural networks. Combining methods of algebraic semantics and nonmonotonic logic, the possibility of integrating both modes of viewing cognition is demonstrated. The main results are (a) that certain activities of connectionist networks can be interpreted as nonmonotonic inferences, and (b) that there is a strict correspondence between the coding of knowledge in Hopfield networks and the knowledge representation in weight-annotated Poole systems. These results show the usefulness of nonmonotonic logic as a descriptive and analytic tool for analyzing emerging properties of connectionist networks. Assuming an exponential development of the weight function, the present account relates to optimality theory-a general framework that aims to integrate insights from symbolism and connectionism. The paper concludes with some speculations about extending the present ideas.

View PDFchevron_right
On the Relationship between I-O Logic and Connectionism
Silvano Colombo Tosatto

2010

In this paper we present an embedding of (a fragment of) Input/Output logic into feed forward Neural Networks. We make use of the neural-symbolic methodology in order to allow neural networks to reason about normative systems. By doing so we are able to exploit normative reasoning within the neural networks setting. We aim at showing how neural networks can be used to represent a knowledge base of Input/Ouput logic rules and to reason about dilemma and contrary to duty problems.

View PDFchevron_right
A Defense of Pure Connectionism
Alex B Kiefer

2019

Connectionism is an approach to neural-networks-based cognitive modeling that encompasses the recent deep learning movement in artificial intelligence. Connectionist models center on statistical inference within neural networks with empirically learnable parameters, which can be represented as graphical models. More recent approaches focus on learning and inference within hierarchical generative models. Contra influential and ongoing critiques, I argue in this dissertation that the connectionist approach to cognitive science possesses in principle (and, as is becoming increasingly clear, in practice) the resources to model even the most rich and distinctly human cognitive capacities, such as abstract, conceptual thought and natural language comprehension and production. Consonant with much previous philosophical work on connectionism, I argue that a core principle—that proximal representations in a vector space have similar semantic values—is the key to a successful connectionist account of the systematicity and productivity of thought, language, and other core cognitive phenomena.

View PDFchevron_right
Connectionism as a Model of Mind
Christian Kaltwasser

Artificial Intelligence is the study of how to make machines behave in an intelligent manner, such as processing information, solving complex problems, or simply providing entertainment. Digital computers are general symbol manipulators capable of quickly applying strict rules to large collections of symbols, to transform these symbols in a way meaningful and useful to people. This sort of "rule-based computation" is one popular model in AI of how machines (including the brain) can behave intelligently. Another model called "connectionism" is based on the interaction of simple processing units combined in a large network, resembling the network of neurons in the human brain. The connectionist approach to intelligence has many features which may make it more appealing as an model of intelligent systems than rule-based computation.

View PDFchevron_right
The connectionist construction of concepts In M
Adrian Cussins

1990

The character of computational modelling of cognition depends on an underlying theory of representation. Classical cognitive science has exploited the syntax/semantics theory of representation that derives from logic. But this has had the consequence that the kind of psychological explanation supported by classical cognitive science is conceptualist: psychological phenomena are modelled in terms of relations that hold between concepts, and between the sensors/effectors and concepts. This kind of explanation is inappropriate for the Proper Treatment of Connectionism . Is there an alternative theory of representation that retains the advantages of classical theory, but which does not force psychological explanation into the conceptualist mould? I outline such an alternative by introducing an experience-based notion of nonconceptual content and by showing how a complex construction out of nonconceptual content can satisfy classical constraints on cognition. The psychologically fundamental structure of cognition is not the structure that holds between concepts, but, rather, the structure within concepts. The theory of the representational structure within concepts allows psychological phenomena to be explained as the progressive emergence of objectivity. This can be modelled computationally by means of the computational processes of a perspective-dependence-reducing transformer. This device may be thought of as a generalisation of a cognitive map, which includes the processes of map-formation and map use. It forms computational structures which take nonconceptual contents as inputs and yield nonconceptual contents as outputs, but do so in a way which makes the resulting capacity of the system less and less dependent on any particular perspectives, yielding satisfactory performance from any point of view.

View PDFchevron_right

References (40)

  1. REFERENCES: Balkenius, C. (1992): "Neural mechanisms for self- organization of emergent schemata, dynamical schema processing, and semantic constraint satisfaction," manuscript, Lund University Cognitive Studies No. 14, Lund University.
  2. Balkenius, C. and P. Gärdenfors (1991): "Nonmonotonic inferences in neural networks," pp. 32-39 in Principles of Knowledge Represen- tation and Reasoning: Proceedings of the Second International Conference, J.A. Allen, R. Fikes, and E. Sandewall, eds. (San Mateo, CA: Morgan Kaufmann).
  3. Beale, R. and T. Jackson (1990): Neural Computing: An Introduction, Bristol: Adam Hilger.
  4. Benthem, J. van (1992): "Logic and the flow of information," to appear in the Proceedings of the 9th International Congress of Logic, Metho- dology, and Philosophy of Science, Amsterdam: North-Holland.
  5. Bernsen, N. O. and I. Ulbaek (1992a): "Two games in Town: Systematicity in distributed connectionist systems", AISBQ Special Issue on Hybrid Models of Cognition Part 2, No. 79, 25-30.
  6. Bernsen, N. O. and I. Ulbaek (1992b): "Systematicity, thought and attention in a distributed connectionist system," manuscript, Centre of Cognitive Science, Roskilde University. processing at the level of entire constituents -i.e., even though the constituents are not causally efficacious."
  7. Churchland, P. S. (1986): Neurophilosophy: Toward a Unified Science of the Mind/Brain, Cambridge, MA: MIT Press.
  8. Cohen, M. A. and S. Grossberg (1983): "Absolute stability of global pattern formation and parallel memory storage by competitive neural networks," IEEE Transactions on Systems, Man, and Cybernetics, SMC-13, 815-826.
  9. Dellarosa, D. (1988): "The psychological appeal of connectionism," Behavioral and Brain Sciences 11:1, 28-29.
  10. Dennett, D. (1978): Brainstorms, Cambridge, MA: MIT Press.
  11. Eijck, J. van, and F. -J. de Vries (1992): "Dynamic interpretation and Hoare deduction," Journal of Logic, Language and Information 1, 1-44.
  12. Fodor, J. (1975): The Language of Thought, Cambridge, MA: Harvard University Press.
  13. Fodor, J. (1981): Representations, Cambridge, MA: MIT Press.
  14. Fodor, J. and B. P. McLaughlin (1990): "Connec- tionism and the problem of systematicity: Why Smolensky's solution doesn't work," Cognition 35, 183-204.
  15. Fodor, J. and Z. Pylyshyn (1988): "Connectionism and cognitive architecture: A critical analysis," in. S. Pinker & J. Mehler (eds.) Connections and Symbols. Cambridge, MA: MIT Press.
  16. Gabbay, D. (1985): "Theoretical foundations for non- monotonic reasoning in expert systems," in Logic and Models of Concurrent Systems, K. Apt ed., Berlin: Springer-Verlag.
  17. Gärdenfors P. (1984): "The dynamics of belief as a basis for logic," British Journal for the Philosophy of Science 35, 1-10.
  18. Gärdenfors, P. (1985): "Propositional logic based on the dynamics of belief," Journal of Symbolic Logic 50, 390-394.
  19. Gärdenfors, P. (1988): Knowledge in Flux: Modeling the Dynamics of Epistemic States, Cambridge, MA: The MIT Press, Bradford Books.
  20. Gärdenfors, P. (1992): "Three levels of inductive inference," to appear in the Proceedings of the 9th International Congress of Logic, Methodology, and Philosophy of Science, Amsterdam: North- Holland.
  21. Gärdenfors, P. and D. Makinson. (1993): "Nonmono- tonic inferences based on expectations," to appear in Artificial Intelligence.
  22. Groenendijk, J. and M. Stokhof (1990): "Dynamic Montague grammar," to appear in Proceedings of the Second Symposium on Logic and Language, Hajduszoboszlo, Hungary.
  23. Groenendijk, J. and M. Stokhof (1991): "Dynamic predicate logic," Linguistics and Philosophy 14, 39-100.
  24. Grossberg, S. (1989): "Nonlinear neural networks: Principles, mechanisms, and architectures," Neural Networks 1, 17-66.
  25. Kraus, S., D. Lehmann, and M. Magidor, (1991), "Nonmonotonic reasoning, preferential models and cumulative logics," Artificial Intelligence 44, 167-207.
  26. Makinson, D. (1993): "General patterns in nonmonotonic reasoning," to appear as Chapter 2 of Handbook of Logic in Artificial Intelligence and Logic Programming, Volume II: Non- Monotonic and Uncertain Reasoning. Oxford: Oxford University Press.
  27. Makinson, D. and P. Gärdenfors (1991): "Relations between the logic of theory change and nonmonotonic logic," pp. 185-205 in The Logic of Theory Change, ed. by A. Fuhrmann and M. Morreau, Springer-Verlag. Also pp. 7-27 in Proceedings of the Workshop on Nonmonotonic Reasoning, GMD 1990 (Arbeitspapiere der GMD 443), ed. by G. Brewka and H. Freitag, Gesellschaft für Mathematik und Daten- verarbeitung MBH, 1990.
  28. Pylyshyn, Z. (1984): Computation and Cognition, Cambridge, MA: MIT Press.
  29. de Rijke, M. (1993): "Meeting some neighbours", in J. van Eijck and A. Visser (eds.) Logic and Information Flow.
  30. Rumelhart, D. E. and J. L. McClelland (1986): Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Cambridge, MA: MIT Press.
  31. Rumelhart, D. E., P. Smolensky, J. L. McClelland and G. E. Hinton (1986): "Schemata and sequential thought processes in PDP models," in Rumelhart, D. E., Parallel Distributed Processing, Vol. 2, pp. 7-57, Cambridge, MA: MIT Press.
  32. Sellars, W. (1980): "Inference and meaning," in J. Sicha ed. Pure Pragmatics and Possible Worlds, Reseda, CA: Ridgeview Publishing Co.
  33. Smolensky, P. (1986): "Information processing in dynamical systems: foundations of harmony theory," in Rumelhart, D.E., Parallel Distributed Processing, Vol. 1, 194-281, Cambridge, MA: MIT Press.
  34. Smolensky, P. (1988): "On the proper treatment of connectionism," Behavioral and Brain Sciences 11, 1-23.
  35. Smolenskty, P. (1991a): "Connectionism, constituency, and the language of thought," in Loewer, B. and Rey G. (eds.), Meaning in Mind: Fodor and His Critics, Oxford: Blackwell, 201- 227.
  36. Smolensky, P. (1991b): "Tensor product variable binding and the representation of symbolic structures in connectionist systems," Artificial Intelligence 46, 159-216.
  37. Thom, R. (1972): Stabilité Structurelle et Morphogénèse, New York: Benjamin.
  38. Veltman, F. (1991): "Defaults in update semantics," in M. Moens (ed), Common sense entailment and update semantics, DYANA-deliverable R2.5.C. Edinburgh, 1991, 2-60, to appear in Journal of Philosophical Logic.
  39. Woodfield, A. and Morton, A. (1988): "The reality of the symbolic and subsymbolic systems," Behavioral and Brain Sciences 11, 58.
  40. Zornetzer, S. F., J. L Davis and C. Lau (1990): An Introduction to Neural and Electronic Networks, San Diego: Academic Press.

Related papers

Connectionism and Information Processing Abstractions
Ashok Goel

Ai Magazine, 1988

View PDFchevron_right
The Grand Challenges and Myths of Neural-Symbolic Computation⋆
Luis Lamb

Recurrent Neural …

View PDFchevron_right
Editorial: Special issue on integration of symbolic and connectionist systems
Franz Kurfess

Cognitive Systems Research, 2002

View PDFchevron_right
On the proper treatment of connectionism
Paul Smolensky

Behavioral and Brain Sciences, 1988

A set of hypotheses is formulated for a connectionist approach to cognitive modeling. These hypotheses are shown to be incompatible with the hypotheses underlying traditional cognitive models. The connectionist models considered are massively parallel numerical computational systems that are a kind of continuous dynamical system. The numerical variables in the system correspond semantically to fine-grained features below the level of the concepts consciously used to describe the task domain. The level of analysis is intermediate between those of symbolic cognitive models and neural models. The explanations of behavior provided are like those traditional in the physical sciences, unlike the explanations provided by symbolic models.Higher-level analyses of these connectionist models reveal subtle relations to symbolic models. Parallel connectionist memory and linguistic processes are hypothesized to give rise to processes that are describable at a higher level as sequential rule appli...

View PDFchevron_right
Phenomena and mechanisms: Putting the symbolic, connectionist, and dynamical systems debate in broader perspective
William Bechtel

Cognitive science is, more than anything else, a pursuit of cognitive mechanisms. To make headway towards a mechanistic account of any particular cognitive phenomenon, a researcher must choose among the many architectures available to guide and constrain the account. It is thus fitting that this volume on contemporary debates in cognitive science includes two issues of architecture, each articulated in the 1980s but still unresolved:

View PDFchevron_right

Related topics

  • Intuitionistic Logic
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved