Academia.eduAcademia.edu

Outline

Title
Abstract
Key Takeaways
Figures
Introduction: The Body and the Brain
Conclusions and Perspectives
References
All Topics
Psychology
Cognitive Science

History and Philosophy of Neural Networks

Profile image of Mark BishopMark Bishop

Abstract

This chapter conceives the history of neural networks emerging from two millennia of attempts to rationalise and formalise the operation of mind. It begins with a brief review of early classical conceptions of the soul, seating the mind in the heart; then discusses the subsequent Cartesian split of mind and body, before moving to analyse in more depth the twentieth century hegemony identifying mind with brain; the identity that gave birth to the formal abstractions of brain and intelligence we know as 'neural networks'. The chapter concludes by analysing this identity-of intelligence and mind with mere abstractions of neural behaviour-by reviewing various philosophical critiques of formal connectionist explanations of 'human understanding', 'mathematical insight' and 'consciousness'; critiques which, if correct, in an echo of Aristotelian insight, suggest that cognition may be more profitably understood not just as a result of [mere abstractions of] neural firings, but as a consequence of real, embodied neural behaviour, emerging in a brain, seated in a body, embedded in a culture and rooted in our world; the so called 4Es approach to cognitive science: the Embodied, Embedded, Enactive, and Ecological conceptions of mind. Contents 1. Introduction: the body and the brain 2. First steps towards modelling the brain 3. Learning: the optimisation of network structure 4. The fall and rise of connectionism 5. Hopfield networks 6. The 'adaptive resonance theory' classifier 7. The Kohonen 'feature-map' 8. The multi-layer perceptron 9. Radial basis function networks 10. Recent developments in neural networks 11. "What artificial neural networks cannot do .." 12. Conclusions and perspectives Glossary Nomenclature References Biographical sketches

Key takeaways
sparkles

AI

  1. Neural networks evolved from philosophical inquiries into the nature of mind and intelligence over two millennia.
  2. The 4Es approach emphasizes cognition as embodied, embedded, enactive, and ecological, challenging traditional neural network models.
  3. Critiques from philosophers like Searle and Penrose argue that computation alone cannot account for human cognition or understanding.
  4. Key historical figures, including McCulloch, Pitts, and Rosenblatt, laid foundational concepts for modern neural networks.
  5. Recent advancements in deep learning and reinforcement learning highlight neural networks' growing importance in AI and cognitive science.
Figures (23)
Total association: in which there is unrestricted association between previous events and/or arbitrary concepts. E.g. James famously describes ‘The memory of a walk followed by a romantic dinner’. Using these principles James offers the following explanation of three types of associa- tive mental processes involved in spontaneous thought: unrestricted association; partial association and focussed association.
Total association: in which there is unrestricted association between previous events and/or arbitrary concepts. E.g. James famously describes ‘The memory of a walk followed by a romantic dinner’. Using these principles James offers the following explanation of three types of associa- tive mental processes involved in spontaneous thought: unrestricted association; partial association and focussed association.
To recall a forgotten item James envisages a situation where two groups of three memories [a,b,c] and [l,m,n] are fully associated (& interconnected) with the ‘forgotten item’ Z. By the process of total association activation of [a,b,c] will eventually propagate to activate [l,m,n] which together [a,b,c, 1,m,n] will serve to activate the forgotten memory Z, see Figure 2.
To recall a forgotten item James envisages a situation where two groups of three memories [a,b,c] and [l,m,n] are fully associated (& interconnected) with the ‘forgotten item’ Z. By the process of total association activation of [a,b,c] will eventually propagate to activate [l,m,n] which together [a,b,c, 1,m,n] will serve to activate the forgotten memory Z, see Figure 2.
Central to the search to resolve this issue was the ability to stain tissue such that it shows up better under microscopes (e.g. see Figure 3), research pioneered by the Bo- hemian anatomist Johannes Evangelista Purkinnje. Johannes worked at the University of Prague where he successfully made the then thinnest known slices of brain tissue; to make thes slices Purkinnje was amongst the first to use a device called a microtome; a tool to cut extremely thin slices of material.
Central to the search to resolve this issue was the ability to stain tissue such that it shows up better under microscopes (e.g. see Figure 3), research pioneered by the Bo- hemian anatomist Johannes Evangelista Purkinnje. Johannes worked at the University of Prague where he successfully made the then thinnest known slices of brain tissue; to make thes slices Purkinnje was amongst the first to use a device called a microtome; a tool to cut extremely thin slices of material.
Activity at the 4, input to an n input neuron is represented by the symbol X; and the effect of the 274, synapse by a weight W;, hence the net effect of the i4, input on the itn Synapse on the MCP cell is thus X; x W;. Thus the ‘modern’ MCP cell (see Figure 4) is denoted as firing if: NB. In a further modern generalisation of the MCP neuron, the MCP output is defined by an arbitrary function of the weighted sum of its input. This function is called the neuron’s activation function. Example activation functions include: linear summation; the Heaviside (unit step or threshold) function, usually denoted by H, which is a discontinuous function whose value is 0 for negative argument and +1 for positive argument; and the oft deployed sigmoid function (which offers a continuous, differentiable, approximation to the Heaviside function).
Activity at the 4, input to an n input neuron is represented by the symbol X; and the effect of the 274, synapse by a weight W;, hence the net effect of the i4, input on the itn Synapse on the MCP cell is thus X; x W;. Thus the ‘modern’ MCP cell (see Figure 4) is denoted as firing if: NB. In a further modern generalisation of the MCP neuron, the MCP output is defined by an arbitrary function of the weighted sum of its input. This function is called the neuron’s activation function. Example activation functions include: linear summation; the Heaviside (unit step or threshold) function, usually denoted by H, which is a discontinuous function whose value is 0 for negative argument and +1 for positive argument; and the oft deployed sigmoid function (which offers a continuous, differentiable, approximation to the Heaviside function).
FIGURE 5. The link between Connectionism and Associationism
FIGURE 5. The link between Connectionism and Associationism
FIGURE 6. The structure of Rosenblatt’s Perceptron
FIGURE 6. The structure of Rosenblatt’s Perceptron
Building upon Bledsoe and Browning’s earlier insights into n-tuple pattern classifica- tion [23], Aleksander noted that if the inputs and outputs to an MCP cell are constrainec to be binary, the MCP cell computes the ‘Binary Discriminant Function’; the operatior of such a [Binary Discriminant] neuron (see Figure 7) being to classify (discriminate) it: binary input data into one of two sets: TRUE or FALSE. Hence a BDN with n binary (0/1) inputs can classify 2” possible binary patterns. For example, a BDN with thre« input lines, (a, b, c) can perform any binary discrimination between 2° = 8 possible binary input pattern vectors (including functions Minsky and Papert had identified as hard for the single layer perceptron such as XOR and parity).
Building upon Bledsoe and Browning’s earlier insights into n-tuple pattern classifica- tion [23], Aleksander noted that if the inputs and outputs to an MCP cell are constrainec to be binary, the MCP cell computes the ‘Binary Discriminant Function’; the operatior of such a [Binary Discriminant] neuron (see Figure 7) being to classify (discriminate) it: binary input data into one of two sets: TRUE or FALSE. Hence a BDN with n binary (0/1) inputs can classify 2” possible binary patterns. For example, a BDN with thre« input lines, (a, b, c) can perform any binary discrimination between 2° = 8 possible binary input pattern vectors (including functions Minsky and Papert had identified as hard for the single layer perceptron such as XOR and parity).
FIGURE 8. The use of a RAM chip as a Binary Discriminant Neuron
FIGURE 8. The use of a RAM chip as a Binary Discriminant Neuron
FIGURE 9. Generalisation in a Logical (or ‘weightless’) neural network sight patterns that will result in maximum FOUR RAMs firing (the four training images plus new unseen vectors defined by OR-ing together input vectors [2+3], [2+4], [3+4], 2+3-+4]); for a comprehensive review of the capabilities, limitations and extensions of the weightless network paradigm see Ludemir et al [70].
FIGURE 9. Generalisation in a Logical (or ‘weightless’) neural network sight patterns that will result in maximum FOUR RAMs firing (the four training images plus new unseen vectors defined by OR-ing together input vectors [2+3], [2+4], [3+4], 2+3-+4]); for a comprehensive review of the capabilities, limitations and extensions of the weightless network paradigm see Ludemir et al [70].
FIGURE 10. Weights in a binary Hopfield network to learn a binary vector representing ‘Mark’ |-1 -1 +1 +1 -1 +1]
FIGURE 10. Weights in a binary Hopfield network to learn a binary vector representing ‘Mark’ |-1 -1 +1 +1 -1 +1]
If the algorithm cannot find a suitable archetype, it creates a new one by adding an ‘uncommitted neuron’ to the recognition layer. The vigilance parameter has considerable influence on the system: higher vigilance (tending to 1.0) produces highly nuanced ‘memories’ (i.e. many, fine-grained categories), while lower vigilance results (around 0.4 or less) in more general, ‘fuzzy’ memories (i.e. fewer, more generic, categories).
If the algorithm cannot find a suitable archetype, it creates a new one by adding an ‘uncommitted neuron’ to the recognition layer. The vigilance parameter has considerable influence on the system: higher vigilance (tending to 1.0) produces highly nuanced ‘memories’ (i.e. many, fine-grained categories), while lower vigilance results (around 0.4 or less) in more general, ‘fuzzy’ memories (i.e. fewer, more generic, categories).
A Kohonen feature map can be considered as a fully inter-connected network of N MCP style cells (defined by a weight vector n), with in addition each neuron also having a further set of weighted inputs to an M element external input vector (defined by the weight vector m), see Figure 12. Like the ART classifier, the Kohonen network also belongs to the class of unsupervised neural networks. In its original form it was significantly biologically inspired. A Kohonen network consists of an array of adaptable neurons - the outputs of which form a ‘feature map’. Over time each neuron becomes ‘tuned’ to a particular input vector, by adaption of its weight vector. After training, similar input vectors will excite similar regions of this ‘feature map’. f% is typically a sigmoid squashing function whose output is always positive. The n internal weights are clamped as a function of the sinc operator and distance, (see Figure 13).
A Kohonen feature map can be considered as a fully inter-connected network of N MCP style cells (defined by a weight vector n), with in addition each neuron also having a further set of weighted inputs to an M element external input vector (defined by the weight vector m), see Figure 12. Like the ART classifier, the Kohonen network also belongs to the class of unsupervised neural networks. In its original form it was significantly biologically inspired. A Kohonen network consists of an array of adaptable neurons - the outputs of which form a ‘feature map’. Over time each neuron becomes ‘tuned’ to a particular input vector, by adaption of its weight vector. After training, similar input vectors will excite similar regions of this ‘feature map’. f% is typically a sigmoid squashing function whose output is always positive. The n internal weights are clamped as a function of the sinc operator and distance, (see Figure 13).
FIGURE 13. The ‘Mexican hat’ operator, or sinc function, defining inter- nal weight values as a function of distance
FIGURE 13. The ‘Mexican hat’ operator, or sinc function, defining inter- nal weight values as a function of distance
FicurRE 14. An illustration of Kohonen network weight adaption with random pairs of real-valued inputs
FicurRE 14. An illustration of Kohonen network weight adaption with random pairs of real-valued inputs
FIGURE 15. The Radial Basis Function network activation of each hidden layer node by a coefficient and sums the resulting values to produce a network output, see Figure 15. A RBF network is defined by two parameters:
FIGURE 15. The Radial Basis Function network activation of each hidden layer node by a coefficient and sums the resulting values to produce a network output, see Figure 15. A RBF network is defined by two parameters:
FIGURE 17. Can a diameter limited Perceptron compute the ‘connect- edness’ predicate Nievergelt identifies the main focus of the book as an analysis of the computational power of the single layer perception (SLP) and reports its two key conclusions as being: (a) that a [single layer] diameter limited perceptron cannot compute the ‘connectedness’ predicate and (b) that a single layer [fixed order] perceptron cannot compute the k-input ‘parity’ (XOR) problem. 11.2. The ‘connectedness’ predicate. A ‘diameter limited perceptron (DLP) is ¢ perceptron in which the inputs to its A-Units must all fall within a receptive field of siz D.
FIGURE 17. Can a diameter limited Perceptron compute the ‘connect- edness’ predicate Nievergelt identifies the main focus of the book as an analysis of the computational power of the single layer perception (SLP) and reports its two key conclusions as being: (a) that a [single layer] diameter limited perceptron cannot compute the ‘connectedness’ predicate and (b) that a single layer [fixed order] perceptron cannot compute the k-input ‘parity’ (XOR) problem. 11.2. The ‘connectedness’ predicate. A ‘diameter limited perceptron (DLP) is ¢ perceptron in which the inputs to its A-Units must all fall within a receptive field of siz D.
FIGURE 18. The [2 x 2]‘blob’ detecting Perceptron 11.3. The ‘order’ of a perceptron. The order of a perceptron is defined as the largest number of inputs to any of its A-Units; for perceptrons to be useful this ‘order’ must be fixed at some value smaller than the size of the retina. Consider a simple problem: to design a perceptron to recognise (fire) if there is one or more groups of [2 x 2] black pixels on its input retina, (see Figure 18).
FIGURE 18. The [2 x 2]‘blob’ detecting Perceptron 11.3. The ‘order’ of a perceptron. The order of a perceptron is defined as the largest number of inputs to any of its A-Units; for perceptrons to be useful this ‘order’ must be fixed at some value smaller than the size of the retina. Consider a simple problem: to design a perceptron to recognise (fire) if there is one or more groups of [2 x 2] black pixels on its input retina, (see Figure 18).
11.4.1. Can an order (1) perceptron solve the odd parity problem? Consider the 3-input odd parity problem, (see Figure 19). An ‘order (1) perceptron’ is functionally equivalent to asimple MCP cell. To solve the odd parity problem an order (1) perceptron with three A-Units (A;, Ag and A3) must fire - output one - when the number of active elements on the retina is odd. FIGURE 19. The 3-input ‘odd parity’ function The desired perceptron response for a retina of size three is depicted in Figure 19. Thus to depict the function described in Figure 19 the perceptron must learn a set of weights, W,, W2, W3 that satisfy the following constrain:
11.4.1. Can an order (1) perceptron solve the odd parity problem? Consider the 3-input odd parity problem, (see Figure 19). An ‘order (1) perceptron’ is functionally equivalent to asimple MCP cell. To solve the odd parity problem an order (1) perceptron with three A-Units (A;, Ag and A3) must fire - output one - when the number of active elements on the retina is odd. FIGURE 19. The 3-input ‘odd parity’ function The desired perceptron response for a retina of size three is depicted in Figure 19. Thus to depict the function described in Figure 19 the perceptron must learn a set of weights, W,, W2, W3 that satisfy the following constrain:
The firing patterns of the twelve A-Units, for the eight possible input patterns (RO .. R7), are shown in Figure 20; where A, is the weight to A-Unit An. FIGURE 20. Response of A-Unit masks to possible input patterns
The firing patterns of the twelve A-Units, for the eight possible input patterns (RO .. R7), are shown in Figure 20; where A, is the weight to A-Unit An. FIGURE 20. Response of A-Unit masks to possible input patterns
FIGURE 21. The two input OR and AND functions are linearly separable
FIGURE 21. The two input OR and AND functions are linearly separable
FIGURE 22. The two input XOR function is not linearly separable
FIGURE 22. The two input XOR function is not linearly separable
FIGURE 23. Mapping the two input XOR function into a 3D space
FIGURE 23. Mapping the two input XOR function into a 3D space

Related papers

Neural Networks and Deep Learning, Charu C. Aggarwal
Eduardo Pablo

Neural Networks and Deep Learning A Textbook, 2018

View PDFchevron_right
Artificial Intelligence & Neural Networks
Patrice S Mubita

One of the greatest mysteries of science is in the elusiveness of knowing exactly how the brain and thus the mind makes thought possible. The phenomenon of unlocking the secrets of the brain and therefore understanding its fundamental areas of function represents one of the greatest challenges of our time. According to most neural network researchers, the objective of AI is to seek an understanding of nature's capabilities for which the human race can engineer solutions to problems that cannot be solved by traditional computing. This paper takes a basic looks at the physical biological brain as the motivation behind AI.

View PDFchevron_right
Artificial Intelligence & Neural Networks II
Patrice S Mubita
View PDFchevron_right
Artificial neural networks explained‐Part 2
Stephen Westland

Journal of the Society of Dyers and Colourists, 1998

View PDFchevron_right
Opening Up the Black Box of Artificial Neural Networks
Jerry Darsey

Journal of Chemical Education, 1994

View PDFchevron_right
Artificial Neural Networks
Ugur Halici

Handbook of Measuring System Design, 2005

View PDFchevron_right
Neural networks: friends or foes?
Stephen Roberts

Sensor Review, 1997

Arti cial neural networks (ANNs), or connectionist systems, are composed of simple non-linear processing units (`neurons') connected into networks. They may be regarded, furthermore, as systems which adapt their functionality as a result of exposure to information (`training'). They have become enormously popular for data analysis over the past decade. Applications have included speech recognition, face recognition, DNA sequence labelling, time series prediction and medical data analysis to name but a few 13, 4, 9]. There are good reasons for this popularity and for hence utilising them, but there are also good reasons for not using them : ANNs o er no`magic' solution to problems and their internal`workings' are often obscure to the user. As such, then, they would appear to be, at best, an unreliable tool and, at worst, downright misleading. Why is it then that they have achieved such notoriety that even hardened statisticians (such as 11]) have become interested in their use? To answer this question we rst consider the two basic forms of data inference, classi cation and regression (prediction is regarded as a subset within the latter).

View PDFchevron_right
Brain, neural networks, and computation
J. Hopfield

Reviews of Modern Physics, 1999

View PDFchevron_right
Naturally intelligent systems Maureen Caudill and Charles Butler, Cambridge, MA, A Bradford Book
Maureen Caudill

Technological Forecasting and Social Change, 1991

View PDFchevron_right
Artificial neural networks explained‐Part 1
Stephen Westland

Journal of the Society of Dyers and Colourists, 1998

View PDFchevron_right

References (147)

  1. Abbot, L., (1990), Learning in Neural Network Memories, Network: Computation in Neural Systems 1, pp. 105-122. Offers a critique of neural modelling in classical connectionist models of low level cognition.
  2. Auer, P, Burgsteiner, H. & Maass, W., (2002), Reducing communication for distributed learning in neural networks, in: J. R. Dorronsoro (Ed.), (2002), Proc. of the International Conference on Arti- ficial Neural Networks ICANN 2002, Vol. 2415 of Lecture Notes in Computer Science, pp. 123-128, Springer. Auer suggests that to approximate a given input/output behaviour F on particular functions of time (or spike trains), it is simply necessary to randomly pick some sufficiently complex reservoir of re- current circuits of spiking neurons, and then merely adapt the weights of a single pool of feedforward spiking neurons to approximate the desired target output behaviour.
  3. Ackley, D.H., Hinton, G,E & Sejnowski, T,J., (1985). A Learning Algorithm for Boltzmann Machines. Cognitive Science: 9 (1), pp. 147?169. First description of Boltzmann machine learning algorithm for neural networks with hidden layers.
  4. Ahmad, S., (1991), VISIT: An efficient computational model of human visual attention, PhD Thesis, University of Illinois, USA. Another critique of neural modelling in classical connectionist models of low level cognition.
  5. Aleksander, I, (1970), Microcircuit learning networks: hamming distance behaviour, Electronics Letters 6 (5). Early introduction to the idea of using small microcircuit devices to implement Bledsoe and Browning's n-tuple recognition systems.
  6. Aleksander, I. & Stonham, T.J., (1979), "Guide to Pattern Recognition using Random Access Mem- ories", Computers & Digital Techniques 2 (1), pp. 29-40, UK. Early review of the n-tuple or weightless network paradigm.
  7. Aleksander, I, Thomas, W.V. & Bowden, P.A., (1984),"WISARD a Radical Step Forward in Image Recognition", Sensor Review, UK. A description of a real-time vision system built using weightless networks.
  8. Aleksander, I & Morton, H., (1995), An Introduction to Neural Computing, Cengage Learning EMEA. Introductory text on neural networks.
  9. Aleksander, I & Morton, H., (2012), Aristotle's laptop: the discovery of our informational mind, World Scientific Publishing, Singapore. Research text outlining Aleksander and Morton's idea of the 'informational mind'.
  10. Bain, A, (1873), Mind and Body. The theories of their relation, D.Appleton & Co, NY. In which Bain suggests that our thoughts and body activity result from interactions among neurons within the brain.
  11. Barto, A.G., Sutton, R. S. & Anderson, C. W. , (1983), Neuron like adaptive elements that can solve difficult learning control problems, IEEE Transactions on Systems, Man and Cybernetics SMC:13: pp. 834-846. The classic introduction to reinforcement learning.
  12. Barnden, J., Pollack, J. (eds.), (1990), High-Level Connectionist Models, Ablex, Norwood NJ, USA. Edited volume which includes critiques of neural modelling in classical connectionist models of high level cognition.
  13. Beer, R. D., (1995), On the dynamics of small continuous-time recurrent neural networks, Adaptive Behaviour 3(4), pp. 469-509.
  14. A key introduction to Continuous Time Recurrent Neural Networks.
  15. Benacerraf, P., (1967), God, the Devil & Gödel, Monist 51. Benacerraf 's response to Lucas's Gödelian argument against materialism.
  16. Bickhard, M. H. (1993). Representational Content in Humans and Machines. Journal of Experimen- tal and Theoretical Artificial Intelligence 5, pp. 285-333.
  17. Mark Bickard highlighting the problem of symbol grounding in cognitive science.
  18. Bickhard, M. H. & Terveen, L.,(1995). Foundational Issues in Artificial Intelligence and Cognitive Science, North Holland. Introduction to Mark Bickhard's interactionist account of cognition.
  19. Bishop, J.M.: Stochastic Searching Networks, Proc. 1st IEE Int. Conf. on Artificial Neural Networks, pp. 329-331, London, UK. First description of the 'Stochastic Diffusion Search' algorithm.
  20. Bishop, J.M., Bushnell, M.J. & Westland, S., (1991), The Application of Neural Networks to Com- puter Recipe Prediction, Color, 16 (1), pp. 3-9. Successful application of neural networks to colour recipe prediction.
  21. Bishop, J.M., Keating, D.A. & Mitchell, R.J., (1998), A Compound Eye for a Simple Robotic Insect, in Austin, J., RAM-Based Neural Networks, pp. 166-174, World Scientific. The use of weightless neural network in mobile robot localisation.
  22. Bishop, J.M., (2002), Dancing with Pixies: strong artificial intelligence and panpyschism. in: Pre- ston, J. & Bishop, J.M., (eds), (2002), Views into the Chinese Room: New Essays on Searle and Artificial Intelligence, Oxford University Press, Oxford UK. First publication of the Dancing with Pixies (DwP) reductio against machine consciousness.
  23. Bishop, J M, (2004), A view inside the Chinese room, Philosopher 28 (4), pp. 47-51. Summary of Searle's Chinese room argument.
  24. Bishop, J.M. (2009), A Cognitive Computation fallacy? Cognition, computations and panpsychism, Cognitive Computation 1(3), pp. 221-233. Critique of the ubiquitous computational metaphor in cognitive science.
  25. Bledsoe, W.W. & Browning, I., (1959), Pattern recognition and reading by machine, Proc. Eastern Joint Computer Conference, pp. 225-232. First description of the n-tuple method for pattern classification later developed into the 'weightless neural network' paradigm by Igor Aleksander.
  26. Bringsjord, S., & Xiao, H., (2000), A refutation of Penrose' Gödelian case against artificial intelli- gence, J. Exp. Theo. Art. Int. 12, pp. 307-329.
  27. Selmer Bringsjord's critique of Roger Penrose's Gödelian argument against artificial intelligence.
  28. Bringsjord, S., & Noel, R., (2002), Real Robots and the Missing Thought-Experiment in the Chinese Room Dialectic. in: Preston, J. & Bishop, J.M., (eds), (2002), Views into the Chinese Room: New Essays on Searle and Artificial Intelligence, Oxford University Press, Oxford UK. Selmer Bringsjord's exposition of a 'missing thought experiment' in Searle's Chinese room argument.
  29. Broomhead, D. S. & Lowe, D., (1988), Radial basis functions, multi-variable functional interpolation and adaptive networks (Technical report). Royal Signals and Radar Establishment RSRE 4148, UK. First exposition of the Radial Basis Function network.
  30. Bryson, A.E., (1963), Optimal programming problems with inequality constraints. I: Necessary conditions for extremal solutions, J. AAAI, 1 (11), pp. 2544-2550. Vapnik cites this paper by Bryson et al as the first publication of the 'back propagation' learning algorithm.
  31. Carpenter, G. & Grossberg, S., (1987), A massively parallel architecture for a self organising neural pattern recognition machine, Computer Vision, Graphics and Image Processing: 37, pp. 54-115. Early publication of the ART-1 classifier.
  32. Chalmers, D., (1990), Why Fodor and Pylyshyn Were Wrong: The Simplest Refutation, In Pro- ceedings of The Twelfth Annual Conference of the Cognitive Science Society, pp. 340-347.
  33. David Chalmers' rebuttal of Fodor and Pylyshyn's critique of connectionism.
  34. Chalmers, D., (1995), Minds, Machines, And Mathematics A Review of Shadows of the Mind by Roger Penrose, PSYCHE, 2: 9. David Chalmers' critique of Roger Penrose's Gödelian argument against artificial intelligence.
  35. Chalmers, D.J., (1996), The Conscious Mind: In Search of a Fundamental Theory, Oxford University Press, Oxford, UK.
  36. David Chalmers' classic monograph espousing his 'psycho-functionalist' theory of mind.
  37. Chrisley R., (1995), Why Everything Doesn't Realize Every Computation, Minds and Machines: Minds and Machines Volume 4, pp. 403-420.
  38. Ron Chrisley addresses Hilary Putnam's critique of functionalism.
  39. Chrisley R., (2006), Counterfactual computational vehicles of consciousness, Toward a Science of Consciousness (April 4th-8th 2006), Tucson Arizona, USA. Ron Chrisley addresses Bishop's 'Dancing with Pixies' reductio against machine consciousness.
  40. Ciresan, D.C., Meier, U. , Gambardella, L. M. Schmidhuber, J., (2010), Deep Big Simple Neural Nets For Handwritten Digit Recognition, Neural Computation 22 (12): pp. 3207?3220. Deep learning networks applied to hand writing recognition.
  41. Copeland B.J., (1997), The broad conception of computation, American Behavioral Scientist 40 (6), pp. 690-716.
  42. Copeland discusses foundations of computing and outlines so called 'real-valued' super-Turing ma- chines.
  43. Cortes, C. & Vapnik, V.N., (1995), Support-Vector Networks, Machine Learning 20 (3), pp. 273 - 297. Early discussion of the Support Vector machine.
  44. Deacon, T, (2012), Incomplete Nature: How Mind Emerged from Matter, W. W. Norton & Com- pany. All encompassing monograph emphasising the importance of appropriate embodiment for the emer- gence of mind.
  45. Dinsmore, J., (1990), Symbolism and Connectionism: a difficult marriage, Technical Report TR 90-8, Southern University of Carbondale, USA. The limitations of uni-variate knowledge representation in classical connectionist systems to arity zero predicates.
  46. Dreyfus, H.L. & Dreyfus, S.E., (1988), Making a mind versus modelling the brain: artificial intelli- gence at a branch point, Artificial Intelligence 117: 1. Historical review of the battle between connectionism and GOFAI in cognitive science.
  47. Fodor, J.A., (1983), Modularity of Mind: An Essay on Faculty Psychology. The MIT Press. In which Fodor postulates a vertical and modular psychological organisation underlying biologically coherent behaviours.
  48. Fodor, J.A., Pylyshyn, Z.W., (1988), Connectionism and Cognitive Architecture: a critical analysis, Cognition 28, pp. 3-72. The classic critique suggesting serious limitations of connectionist systems in comparison to symbol processing systems.
  49. Freeman, A. (2003), Output still not really convinced, The Times Higher (April 11th 2003), UK. ' Discussion of the Chinese room argument.
  50. Freeman, A., (2004), The Chinese Room Comes of Age: a review of Preston & Bishop, Journal of Consciousness Studies 11 (5/6), pp. 156-158. Discussion of the Chinese room argument and Preston/Bishop's edited text.
  51. Fukushima, K, (1980), Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics 36 (4), pp. 193?202. The Neocognitron: one of the first 'deep learning' connectionist systems.
  52. Gardenfors, P, (2004), Conceptual Spaces as a Framework for Knowledge Representation, Mind and Matter 2 (2), pp. 9-27. . On the problem of modelling representations.
  53. Garvey, J., (2003), A room with a view?, The Philosophers Magazine (3rd Quarter 2003), p. 61. Discussion of the Chinese room argument and Preston/Bishop's edited text.
  54. Gerstner, W. & Kistler, W., (2002), Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge University Press, UK. Classic text on spiking neuron connectionist modelling.
  55. Gibson, J.J., (1979), The Ecological Approach to Visual Perception. Boston: Houghton Mifflin. Gibson's classical text outlining the radical 'ecological approach' to vision and visual perception.
  56. Grossberg, S., (1976), Adaptive Pattern Classification and Universal recording: II. Feedback, ex- pectation, olfaction, and illusions, Bio. Cybernetics 23: pp. 187-202. Early exposition of the ART classifier.
  57. Harnad S., (1990), The Symbol Grounding Problem. Physica D 42, pp. 335-346. Harnad's key reference on the classical 'symbol grounding' problem.
  58. Harnish, S., (2001), Minds, Brains, Computers: An Historical Introduction to the Foundations of Cognitive Science, Wiley-Blackwell. Excellent general introductory text to Cognitive Science.
  59. Hebb, D.O. (1949). The Organisation of Behaviour. New York: Wiley & Sons. The original exposition of Hebbian learning.
  60. Hinton, G, (2007), Learning multiple layers of representation, Trends in Cognitive Science 11 (10), pp. 428-434.
  61. Hinton's discussion of deep learning.
  62. Hodgkin, A. L. & Huxley, A. F., (1952), A quantitative description of membrane current and its application to conduction and excitation in nerve. Journal of Physiology 117 (4), pp. 500?544. The classic paper on the detailed Hodgkin-Huxley model of the [squid] neuron.
  63. Hopfield, J..J., (1982), Neural networks and physical systems with emergent collective computational abilities, Proc. Nat. Acad. Sci. 79 (8), pp. 2554-2558. Outlines the use of methods from physics to analyse properties of neural systems.
  64. Hopfield, J. J. (1984) Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Nat. Acad. Sci. 81, pp. 3088-3092. The binary Hopfield neural network.
  65. Hopfield, J. J. and Tank, D. W., (1985), Neural computation of decisions in optimization problems. Biological Cybernetics 55, pp. 141-146. The use of Hopfield networks in optimisation.
  66. Hubel, D.H., Wiesel, T.N. (1962), Receptive fields, binocular interaction and functional architecture in the cat's visual cortex, Journal of Physiology 160 (1), pp, 106-154.
  67. Hubel & Wiesel's classic paper on neuron behaviour.
  68. Irwin, G., Warwick, K. & Hunt, K., (eds), (1995), Neural Network Applications In Control. IET Digital Library, UK. A selection of neural network applications.
  69. Jaeger , H. & Haas, H. (2004), Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication, Science 304: 5667, pp. 78-80. On why echo state networks learn very efficiently.
  70. James, W, (1891), Psychology (briefer course), Holt, New York, USA. Early classic text on the principles of psychology.
  71. Katevas, N., Sgouros, N.M., Tzafestas, S., Papakonstantinou G., Beattie, P., Bishop, J.M., Tsanakas, P & Koutsouris D., (1997), The Autonomous Mobile Robot SENARIO: A Sensor-Aided Intelligent Navigation for Powered Wheelchairs, IEEE Robotics and Automation 4 (4), pp. 60-70. The use of Stochastic Diffusion in mobile robot localisation.
  72. Kleene, S. C., (1956), Representation of events in nerve nets and finite automata. In Automata Studies 34, Annals of Mathematics Studies, pp. 3?42. Princeton University Press, Princeton, NJ.
  73. Before Minsky, Kleene proved that rational-weighted recurrent neural networks with boolean activation functions are computationally equivalent to classical finite state automata.
  74. Klein, C., (2004), Maudlin on Computation, (working paper). On Maudlin's model of the implementation of computation.
  75. Kohonen, T., (1988), Self-Organising Maps, Springer. Introduction to Kohonen feature maps.
  76. Kohonen, T, Barna, G. & Chrisley, R., (1988), Statistical Pattern Recognition with Neural Net- works, In: Proceedings of the IEEE Conference on Neural Network 1: 61-68, 24-27 July 1988, San Diego, CA. A discussion of practical issues relating to the Boltzmann machine and comparison of performance with other neural architectures.
  77. Le Cun, Y., (1985), Une procedure d'apprentissage pour reseau a seuil asymmetrique (a Learn- ing Scheme for Asymmetric Threshold Networks), Proceedings of Cognitiva 85, pp. 599-604, Paris, France. Independent derivation of the back propagation learning algorithm.
  78. Lighthill, J., (1973), Artificial Intelligence: A General Survey, in Artificial Intelligence: a paper symposium, Science Research Council, UK. UK government-funded study providing a critique of GOFAI methods.
  79. Lucas, J.R., (1961), Minds, Machines & Gödel, Philosophy, 36. Lucas outlines the use of Gödelian argument against materialism.
  80. Ludermir, T.B., de Carvalho, A., Braga, A.P. & de Souto, M.C.P., (1999), Weightless neural models: a review of current and past works, Neural Computing Surveys 2, pp. 41-61. Recent review of the weightless neural network paradigm.
  81. Maass, W. & Bishop, C., (eds.), (1999), Pulsed Neural Networks, The MIT Press, Cambridge MA, USA. Classic text on spiking neural networks.
  82. Maass, W, Natschlager, T. & Markram, H., (2002), Real-time computing without stable states: A new framework for neural computation based on perturbations, Neural Computation 14 (11), pp. 2531-2560. Learning rules for Liquid State machines.
  83. Maass, W. & Markram, H., (2004), On the computational power of circuits of spiking neurons, Journal of Computer and System Sciences 69 (4), pp. 593?616. On the use of the same connectionist hardware to compute different functions.
  84. Maudlin, T., (1989), Computation and Consciousness, Journal of Philosophy 86, pp. 407-432. On the execution of computer programs.
  85. McCarthy, J., Minsky, M., Rochester, N. & Shannon, C., (1955), A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence. In: McCarthy, J., Minsky, M., Rochester, N. & Shannon, C., (2006) A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence August 31 1955, AI MAgazine 27(4): 12-14. Reprint of the proposal for the 1956 Dartmouth Conference, along with the short autobiographical statements of the proposers.
  86. McCorduck, P., (1979), Machines who think, Freeman & Co., NY. An account of the Dartmouth Conference.
  87. McCulloch, W.S., Pitts, W., (1943), A logical calculus immanent in nervous activity, Bulletin of Mathematical Biophysics 5, pp. 115-133. On the computational power of the McCulloch-Pitts neuron model.
  88. Minsky, M., (1954), Neural Nets and the brain model problem. PhD thesis, Princeton, USA. Early work from Minsky on a connectionist theory of mind.
  89. Minsky, M. L. (1967). Computation: finite and infinite machines. Prentice-Hall, Inc., Upper Saddle River, NJ, USA. After Kleene, Minsky proved that rational-weighted recurrent neural networks with boolean activation functions are computationally equivalent to classical finite state automata.
  90. Minsky, M & Papert, S., (1969), Perceptions: an introduction to computational geometry, The MIT Press. Devastatining critique of the power of single layer perceptrons.
  91. Mumford, D., (1994), Neural Architectures for Pattern-theoretic Problems. In: Koch, Ch., Davies, J.L. (eds.), (1994), Large Scale Neuronal Theories of the Brain. The MIT Press, Cambridge MA. In which Mumford observes that systems which depend on interaction between feedforward and feed- back loops are quite distinct from models based on Marr's feedforward theory of vision.
  92. Nasuto, S.J., Bishop, J.M., Lauria, S., (1998), Time Complexity Analysis of the Stochastic Diffusion Search, Neural Computation '98, Vienna, Austria. Derivation of the computational time complexity of stochastic diffusion search.
  93. Nasuto, S.J., Bishop, J.M.:, (1998), Neural Stochastic Diffusion Search Network -a theoretical solution to the binding problem, Proc. ASSC2, pp. 19-20, Bremen, Germany. Examination of the binding problem through the lens of stochastic diffusion search.
  94. Nasuto, S.J., Dautenhahn K., Bishop, J.M., (1999), Communication as an Emergent Metaphor for Neuronal Operation. In: Nehaniv, C., (ed), (1999), Computation for Metaphors, Analogy, and Agents, Lecture Notes in Artificial Intelligence 1562, pp. 365-379. Springer. Outlines a new metaphor grounded upon stochastic diffusion search for neural processing.
  95. Nasuto, S.J., Bishop, J.M., (1999), Convergence of the Stochastic Diffusion Search, Parallel Algo- rithms 14 (2), pp. 89-107. Proof of the convergence of stochastic diffusion search to global optimum.
  96. Nasuto, S.J., Bishop, J.M. & De Meyer (2009), Communicating neurons: a connectionist spiking neuron implementation of stochastic diffusion search, Neurocomputing 72: pp. 4-6, pp. 704-712. Review of a communication metaphor -based upon stochastic diffusion search -for neural processing.
  97. Newell, A., & Simon, H.A., (1976), Computer science as empirical inquiry: symbols and search, Communications of the ACM 19 (3), pp. 113-126. Outlines the Physical Symbol System Hypothesis.
  98. Ng, A. & Dean, J., (2012), Building High-level Features Using Large Scale Unsupervised Learning, Proc. 29th Int. Conf. on Machine Learning, Edinburgh, Scotland, UK. Deep learning neural network that successfully learned to recognise higher-level concepts (e.g. human faces and cats) from unlabelled video streams.
  99. Newell, A., Shaw, J.C. & Simon, H.A., (1960), A variety of intelligent learning in a General Problem Solver, in Yovits, M.C & Cameron, S., (eds), 'Self Organising Systems', Pergamon Press, NY. Early discussion of the General Problem Solver.
  100. Overill, J., (2004), Views into the Chinese Room: New Essays on Searle and Artificial Intelligence, Journal of Logic and Computation 14 (2), pp. 325-326. Critique of the Chinese room argument.
  101. Penrose, R., (1989), The Emperor's New Mind: Concerning Computers, Minds, and the Laws of Physics, Oxford University Press, Oxford UK.
  102. Roger Penrose's text criticising computational artificial intelligence from a Gödelian perspective.
  103. Penrose, R., Shadows of the Mind: A Search for the Missing Science of Consciousness, Oxford University Press, Oxford UK.
  104. Roger Penrose's second text criticising computational artificial intelligence from a Gödelian perspec- tive.
  105. Penrose, R., Beyond the Doubting of a Shadow A Reply to Commentaries on Shadows of the Mind, PSYCHE, 2 (23). Roger Penrose replies to criticism of his Gödelian arguments.
  106. Pinker, S. & Prince, A., (1988), On Language and Connectionism: Analysis of a Parallel Distributed Processing Model of Language Acquisition. In: Pinker, S., Mahler, J. (eds.), (1988), Connections and Symbols, The MIT Press, Cambridge MA. Deficiencies in the representation of complex knowledge by classical neural networks .
  107. Preston, J. & Bishop, J.M., (eds), (2002), Views into the Chinese Room: New Essays on Searle and Artificial Intelligence, Oxford University Press, Oxford UK. Edited collection of essays on John Searle's Chinese room argument.
  108. Psyche, Symposium on Roger Penrose's Shadows of the Mind, http://psyche.cs.monash.edu.au/psyche-index-v2.html, PSYCHE VOL 2. . Special issues on Roger Penrose's Gödelian arguments against AI.
  109. Putnam, H., (1988), Representation & Reality, Bradford Books, Cambridge MA. The appendix contains Putnam's refutation of functionalism.
  110. Rapaport, W.J., (2006), Review of Preston, J. & Bishop, J.M., (eds), (2002), Views into the Chinese Room: New Essays on Searle and Artificial Intelligence, Oxford University Press, Oxford UK. In: Australian Journal of Philosophy, 94(1), pp. 129-145. Review of work on the Chinese room argument.
  111. Rey, G, (2002), Searle's Misunderstanding of Functionalism and Strong AI. In: Preston, J. & Bishop, J.M., (eds), (2002), Views into the Chinese Room: New Essays on Searle and Artificial Intelligence, Oxford University Press, Oxford UK. Criticism of Searle's Chinese room argument.
  112. Richeimer, J., (2004), Review of Preston, J. & Bishop, J.M., (eds), (2002), Views into the Chinese Room: New Essays on Searle and Artificial Intelligence, Oxford University Press, Oxford UK, Philo- sophical Books 45 (2), pp.162-167. Review of work on the Chinese room argument.
  113. Rosenblatt, F, (1958), Two theorems of statistical separability in the Perceptron. In: The Mecha- nisation of thought processes: proceedings of a symposium held at the National Physical Laboratory 1, pp. 419-449, HMSO London. Early work by Frank Rosenblatt on the power of the Perceptron.
  114. Rosenblatt, F, (1962), The Principles of Neurodynamics, Spartan Books, New York. Introduction to Perceptrons.
  115. Rumelhart, D.E., McClelland, J.L.., (1986), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, The MIT Press, Cambridge MA. Seminal text which helped re-awaken interest in neural networks post Minsky & Papert's critique.
  116. Rumelhart, D.E.., Hinton, G.E. & Williams, R.J., (1986), Learning representations by back- propagating errors, Nature 323 (6088), pp. 533-536. Independent derivation of the back-propagation algorithm.
  117. Russel, B., (1905), On Denoting, Mind XIV(4), pp. 479-493, Oxford UK. On quantified logical inference.
  118. Schank, R C & Abelson, R P, (1977), Scripts, plans, goals and understanding: An inquiry into human knowledge structures, Psychology Press. On the 'machine understanding' of stories.
  119. Searle, J. R., (1980), Minds, brains, and programs. Behavioral and Brain Sciences 3 (3): 417-457, Cambridge, UK. The Chinese room argument.
  120. Searle, J., (1990), Is the Brain a Digital Computer?, Proceedings of the American Philosophical Association 64, pp.21-37. Is computational syntax reducible to physics?.
  121. Searle, J., (1992), The Rediscovery of the Mind, pp. 227, MIT Press, Cambridge MA. On computations and mind.
  122. Searle, J., (1994), The Mystery of Consciousness, Granta Books, London UK. Interviews and reflections with a selection of twentieth century thinkers on consciousness.
  123. Sejnowski, T.J., Rosenberg, C.R., (1987), Parallel networks that learn to pronounce English text. Complex Systems 1, pp. 145-168. Example of an ANN which learns to pronounce English text.
  124. Siegelmann, H. T., (1993), Neural and Super-Turing Computing, Minds and Machines 13, pp. 103?114. On Siegelmann's claim that recurrent neural networks have super-Turing computational power.
  125. Siegelmann, H. T., (1995), Computation Beyond the Turing Limit, Science 268 (5210), pp. 545-548. On Siegelmann's claim that recurrent neural networks have super-Turing computational power.
  126. Siegelmann, H.T & Sontag, E.F., (1984), Analog computation via neural networks, Theoretical Computer Science 131, pp. 331-360. On Siegelmann's claim that recurrent neural networks have super-Turing computational power.
  127. Smith P., (1998), Explaining Chaos. Cambridge University Press, Cambridge, UK. Includes discussion of the 'shadowing theorems' in chaos.
  128. Smolensky, P., (1988), Connectionist Mental States: A Reply to Fodor and Pylyshyn , Southern Journal of Philosophy 26 (supplement), pp.137-161.
  129. A refutation of Fodor and Pylyshyn's criticism of connectionism.
  130. Sprevak, M.D., (2005), The Chinese Carnival, Studies in the History & Philosophy of Science 36, pp. 203-209. Reflections on Searle's Chinese room argument.
  131. Tanay, T., Bishop, J.M., Spencer, M.C., Roesch, E.B. & Nasuto, S.J., (2013), Stochastic Diffusion Search applied to Trees: a Swarm Intelligence heuristic performing Monte-Carlo Tree Search. In: Bishop, J.M & Erden Y.J, (2013), (eds), Proceedings of the AISB 2013: Computing and Philosophy symposium,'What is computation?', Exeter, UK. Describes the use of a swarm intelligence paradigm -stochastic diffusion search -to play the strategic game of HeX.
  132. Tesauro, G. 1989. Neurogammon wins Computer Olympiad. Neural Computing 1, pp. 321-323. Playing backgammon using neural networks.
  133. Tarassenko, L., (1998), A Guide to Neural Computing Applications, Arnold, London. Practical introduction to neural computing.
  134. Tassinari, R.P. & D'Ottaviano, I.M.L., (2007), Cogito ergo sum non machina! About Gödel's first incompleteness theorem and Turing machines, CLE e-Prints, Vol. 7 (3). Discussion of Penrose arguments against artificial intelligence.
  135. Thompson, E, (2010), Mind in Life, Belknap Press. Thompson's claim that mind is causally related to life.
  136. Torrance, S., Thin Phenomenality and Machine Consciousness, in R.Chrisley, R.Clowes and S.Torrance, (eds), (2005), Proc. 2005 Symposium on Next Generation Approaches to Machine Consciousness: Imagination, Development, Intersubjectivity and Embodiment, AISB05 Convention. Hertfordshire: University of Hertfordshire, pp. 59-66. On differences between brain simulations and real brains.
  137. Touzet, C., (1997), Neural Reinforcement Learning for Behaviour Synthesis. in: N. Sharkey, N., (ed) (1997), Special issue on Robot Learning: The New Wave, Journal of Robotics and Autonomous Systems 22 (3/4), pp. 251-281. In reinforcement learning Touzet details replacing the Q-table with an MLP network or Kohonen network.
  138. Turing, A. M., (1937), [Delivered to the Society in November 1936], "On Computable Numbers, with an Application to the Entscheidungsproblem". Proceedings of the London Mathematical Society. 2 42. pp. 23-65, London, UK. Introduction to the Turing machine and its application to the Entscheidungsproblem.
  139. Van Gelder, T., Port, R.F. (1995), It's About Time: an overview of the dynamical approach to cognition. In: Port, R.F. & Van Gelder, T. (eds.), (1995), Mind as Motion, pp. 1-45, The MIT Press, Cambridge MA. Introduction to the dynamical system theory of mind.
  140. Van de Velde, F., (1997), On the use of computation in modelling behaviour, Network: Computa- tion in Neural Systems 8, pp. 1-32. Critique of neural modelling in classical connectionist models of low level cognition.
  141. Varela, F.J., Thompson, E. & Roesch, E., (1991), The Embodied Mind: Cognitive Science and Human Experience. The MIT Press. Varela et al's seminal introduction to the so called 'embodied concept' of mind.
  142. Waskan, Jonathan A., (2005), Review of Preston, J. & Bishop, J.M., (eds), (2002), Views into the Chinese Room: New Essays on Searle and Artificial Intelligence, Oxford University Press, Oxford UK. Philosophical Review, 114: 2, (2005). Further reflections on the Chinese room argument.
  143. Werbos, P., (1974), Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences, PhD thesis, Harvard University. 'Usually' identified as the first derivation of the back propagation algorithm.
  144. Wheeler, M, (2002), Change in the Rules: Computers, Dynamical Systems, and Searle. In: Preston, J. & Bishop, J.M., (eds), (2002), Views into the Chinese Room: New Essays on Searle and Artificial Intelligence, Oxford University Press, Oxford UK. In which Mike Wheeler uses the Chinese room argument to target all purely formal theories of mind.
  145. Whitehead A.N. & Russell, B., (1910), Principia Mathematica (3 volumes), Cambridge University Press, Cambridge UK. An attempt to derive mathematics from 'first principles'.
  146. Wittgenstein, L., (1948), Last Writings on the Philosophy of Psychology (vol. 1). Blackwell, Oxford, UK. The writings Wittgenstein composed during his stay in Dublin between October 1948 and March 1949, one of his most fruitful periods, which informed the Philosophical Investigations. Biographical sketches
  147. Dr. Mark Bishop is Professor of Cognitive Computing at Goldsmiths, University of London; Director of the Goldmsiths centre for Radical Cognitive Science and Chair of the UK society for Artificial Intelligence and the Simulation of Behaviour (AISB). He has published over 140 articles in the field of Cognitive Computing: its theory, where his

Related papers

Review of James A. Anderson and Edward Rosenfeld (eds.) Talking Nets: An Oral History of Neural Networks (Cambridge, MA and London: MIT Press/Bradford Books, 1998) Annals of Science 62, no. 1 (2005): 130-132.
Tara Abraham
View PDFchevron_right
Fascinating world of Neural Networks
Swedha B
View PDFchevron_right
Neural networks and the human mind: new mathematics fits humanistic insight
Paul Werbos

[Proceedings] 1992 IEEE International Conference on Systems, Man, and Cybernetics, 1992

The past two years have seen substantial progress in artificial neural networks(ANNs). New ANN control designs, combining reinforcement learning and generalized backpropagation[l,21, have demonstrated success on large-scale d-world problems which could not be solved by earlier designs, neural or nonneural. "here now exists a ladder of designs, rising up from simple designs (of limited capability, but a good starting point), through to complex proven designs (which give more power and flexibility after they are mastered), up to new untested designs and ideas which should be able to replicate and explain human intelligence at the highest possible level. After a brief review of neurocontrol, and an explanation of reinforcement learning, this paper asks what imdigitions these designs have for our understanding of the human mind. It argues that this new mathematics is fully compatible with older deep insights into the human mind, due to humanistic thinkers East and West. One m a y therefore hope that this new view of the human mind may be of value in unifying important strands of human culture. From an engineering point of view, the human brain is simply a computer, an information processing system. The function of any computer, as a whole system, is to compute its outputs. The outputs of the brain are control signals to muscles and glands. Therefore the brain as a whole svstem is a neurocontroller (a neural net control system) [3]. To understand the brain in functional, mathematical terms we should therefore focus on the subject of neurocontrol.

View PDFchevron_right
Neural networks: I. Theoretical unification through connectionism
Warren Tryon

Clinical Psychology Review, 1993

.

View PDFchevron_right
Artificial Neural Networks: a Bio-Inspired Revolution or a Long Lasting Misconception and Self-Delusion
Emanuel Diamant

2018

Ali Rahimi, best paper award recipient at NIPS 2017, labelled the current state of Deep Learning (DL) headway as “alchemy”. Yann LeCun, one of the prominent figures in the DL R&D, was insulted by this expression. However, in his response, LeCun did not claimed that DL designers know how and why their DL systems reach so surprising performances. The possible reason for this cautiousness is: No one knows how and in which way system input data is transformed into semantic information at the system’s output. And this, certainly, has its own reason: No one knows what information is! I dare to offer my humble clarification about this obscure and usually untouchable matter. I hope someone would be ready to line up with me.

View PDFchevron_right

Related topics

  • Psychology
  • Cognitive Science
  • Embodied Cognition
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved