Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Results
Discussion
References
All Topics
Engineering
Bioengineering

Modelling of syntactical processing in the cortex

Profile image of Günther PalmGünther Palm

2007, Biosystems

https://doi.org/10.1016/J.BIOSYSTEMS.2006.04.027
visibility

…

description

16 pages

link

1 file

Abstract

Probably the hardest test for a theory of brain function is the explanation of language processing in the human brain, in particular the interplay of syntax and semantics. Clearly such an explanation can only be very speculative, because there are essentially no animal models and it is hard to study detailed neural processing in humans. The approach presented in this paper uses well established basic neural mechanisms in a plausible global network architecture that is formulated essentially in terms of cortical areas and their intracortical and corticocortical interconnections. The neural implementation of this system shows that the comparatively intricate logical task of understanding semantico-syntactical structures can be mastered by a neural network architecture. The system presented also shows additional context awareness, in particular the model is able to correct ambiguous input to a certain degree, e.g. the input "bot show/lift green wall" with an artificial ambiguity between "show" and "lift" is correctly interpreted as "bot show green wall" since a wall is not liftable. Furthermore, the system is able to learn new object words during runtime.

Related papers

A Neurobiologically Constrained Cortex Model of Semantic Grounding With Spiking Neurons and Brain-Like Connectivity
Max Garagnani

One of the most controversial debates in cognitive neuroscience concerns the cortical locus of semantic knowledge and processing in the human brain. Experimental data revealed the existence of various cortical regions relevant for meaning processing, ranging from semantic hubs generally involved in semantic processing to modality-preferential sensorimotor areas involved in the processing of specific conceptual categories. Why and how the brain uses such complex organization for conceptualization can be investigated using biologically constrained neurocomputational models. Here, we improve pre-existing neurocomputational models of semantics by incorporating spiking neurons and a rich connectivity structure between the model 'areas' to mimic important features of the underlying neural substrate. Semantic learning and symbol grounding in action and perception were simulated by associative learning between co-activated neuron populations in frontal, temporal and occipital areas. As a result of Hebbian learning of the correlation structure of symbol, perception and action information, distributed cell assembly circuits emerged across various cortices of the network. These semantic circuits showed category-specific topographical distributions, reaching into motor and visual areas for action-and visually-related words, respectively. All types of semantic circuits included large numbers of neurons in multimodal connector hub areas, which is explained by cortical connectivity structure and the resultant convergence of phonological and semantic information on these zones. Importantly, these semantic hub areas exhibited some category-specificity, which was less pronounced than that observed in primary and secondary modality-preferential cortices. The present neurocomputational model integrates seemingly divergent experimental results about conceptualization and explains both semantic hubs and category-specific areas as an emergent process causally determined by two major factors: neuroanatomical connectivity structure and correlated neuronal activation during language learning.

View PDFchevron_right
On the structural ambiguity in natural language that the neural architecture cannot deal with
Rens Bod

Behavioral and Brain Sciences, 2006

We argue that van der Velde's & de Kamps's model does not solve the binding problem but merely shifts the burden of constructing appropriate neural representations of sentence structure to unexplained preprocessing of the linguistic input. As a consequence, their model is not able to explain how various neural representations can be assigned to sentences that are structurally ambiguous.

View PDFchevron_right
An Associative Cortical Model of Language Understanding and Action Planning
Günther Palm

Lecture Notes in Computer Science, 2005

The brain representations of words and their referent actions and objects appear to be strongly coupled neuronal assemblies distributed over several cortical areas. In this work we describe the implementation of a cell assembly-based model of several visual, language, planning, and motor areas to enable a robot to understand and react to simple spoken commands. The essential idea is that different cortical areas represent different aspects of the same entity, and that the longrange cortico-cortical projections represent hetero-associative memories that translate between these aspects or representations.

View PDFchevron_right
Neurally and mathematically motivated architecture for language and thought
Leonid Perlovsky

The open neuroimaging journal, 2010

Neural structures of interaction between thinking and language are unknown. This paper suggests a possible architecture motivated by neural and mathematical considerations. A mathematical requirement of computability imposes significant constraints on possible architectures consistent with brain neural structure and with a wealth of psychological knowledge. How language interacts with cognition. Do we think with words, or is thinking independent from language with words being just labels for decisions? Why is language learned by the age of 5 or 7, but acquisition of knowledge represented by learning to use this language knowledge takes a lifetime? This paper discusses hierarchical aspects of language and thought and argues that high level abstract thinking is impossible without language. We discuss a mathematical technique that can model the joint language-thought architecture, while overcoming previously encountered difficulties of computability. This architecture explains a contra...

View PDFchevron_right
How Does the Brain Represents Language and Answers Questions? Using an AI System to Understand the Underlying Neurobiological Mechanisms
Marco idiart

Frontiers in Computational Neuroscience

View PDFchevron_right
Brain basis of meaning, words, constructions, and grammar
Bert Cappelle
View PDFchevron_right
An Associative Model of Cortical Language and Action Processing
Günther Palm

Modeling Language, Cognition and Action - Proceedings of the Ninth Neural Computation and Psychology Workshop, 2005

The brain correlates of words and their referent actions and objects appear to be strongly coupled neuron ensembles or assemblies distributed over defined cortical areas. In this work we describe the implementation of a cell assembly-based model of several visual, language, planning, and motor areas to enable a robot to understand and react to simple spoken commands. The essential idea is that different cortical areas represent different aspects of the same entity, and that the long-range cortico-cortical projections represent hetero-associative memories that translate between these aspects or representations.

View PDFchevron_right
How the Brain Represents Language and Answers Questions? Using an AI System to Understand the Underlying Neurobiological Mechanisms
Marco idiart, Aline Villavicencio

Frontiers in Computational Neuroscience, 2019

To understand the computations that underlie high-level cognitive processes we propose a framework of mechanisms that could in principle implement START, an AI program that answers questions using natural language. START organizes a sentence into a series of triplets, each containing three elements (subject, verb, object). We propose that the brain similarly defines triplets and then chunks the three elements into a spatial pattern. A complete sentence can be represented using up to 7 triplets in a working memory buffer organized by theta and gamma oscillations. This buffer can transfer information into long-term memory networks where a second chunking operation converts the serial triplets into a single spatial pattern in a network, with each triplet (with corresponding elements) represented in specialized subregions. The triplets that define a sentence become synaptically linked, thereby encoding the sentence in synaptic weights. When a question is posed, there is a search for the closest stored memory (having the greatest number of shared triplets). We have devised a search process that does not require that the question and the stored memory have the same number of triplets or have triplets in the same order. Once the most similar memory is recalled and undergoes 2-level dechunking, the sought for information can be obtained by element-by-element comparison of the key triplet in the question to the corresponding triplet in the retrieved memory. This search may require a reordering to align corresponding triplets, the use of pointers that link different triplets, or the use of semantic memory. Our framework uses 12 network processes; existing models can implement many of these, but in other cases we can only suggest neural implementations. Overall, our scheme provides the first view of how language-based question answering could be implemented by the brain. Keywords: episodic memory; memory retrieval; question and answer; short-term (working) memory; theta-gamma code.

View PDFchevron_right
A Spiking Neural Network Model of Multi-modal Language Processing of Robot Instructions
Christo Panchev

Lecture Notes in Computer Science, 2005

Presented is a spiking neural network architecture of human language instruction recognition and robot control. The network is based on a model of a leaky Integrate-And-Fire (lIAF) spiking neurone with Active Dendrites and Dynamic Synapses (ADDS) [1-3]. The architecture contains several main modules associating information across different modalities: an auditory system recognising single spoken words, a visual system recognising objects of different colour and shape, a motor control system for navigation and motor control and a working memory. The main focus of this chapter is the working memory module whose function is sequential processing of words from a language instruction, task and goal representation and cross-modal association of objects and actions. We test the model with a robot whose goal is to recognise and execute language instructions. The work demonstrates the potential of spiking neurons for processing spatio-temporal patterns and the experiments present spiking neural networks as a paradigm which can be applied for modelling sequence detectors at word level for robot instructions.

View PDFchevron_right
Tracking neural coding of perceptual and semantic features of concrete nouns
Dean Pomerleau, Tom Mitchell, Alona Fyshe

NeuroImage, 2012

We present a methodological approach employing magnetoencephalography (MEG) and machine learning techniques to investigate the flow of perceptual and semantic information decodable from neural activity in the half second during which the brain comprehends the meaning of a concrete noun. Important information about the cortical location of neural activity related to the representation of nouns in the human brain has been revealed by past studies using fMRI. However, the temporal sequence of processing from sensory input to concept comprehension remains unclear, in part because of the poor time resolution provided by fMRI. In this study, subjects answered 20 questions (e.g. is it alive?) about the properties of 60 different nouns prompted by simultaneous presentation of a pictured item and its written name. Our results show that the neural activity observed with MEG encodes a variety of perceptual and semantic features of stimuli at different times relative to stimulus onset, and in different cortical locations. By decoding these features, our MEG-based classifier was able to reliably distinguish between two different concrete nouns that it had never seen before. The results demonstrate that there are clear differences between the time course of the magnitude of MEG activity and that of decodable semantic information. Perceptual features were decoded from MEG activity earlier in time than semantic features, and features related to animacy, size, and manipulability were decoded consistently across subjects. We also observed that regions commonly associated with semantic processing in the fMRI literature may not show high decoding results in MEG. We believe that this type of approach and the accompanying machine learning methods can form the basis for further modeling of the flow of neural information during language processing and a variety of other cognitive processes.

View PDFchevron_right

References (72)

  1. Abeles, M., 1991. Corticonics: Neural Circuits of the Cerebral Cortex. Cambridge University Press, Cambridge, UK.
  2. Abeles, M., Bergman, H., Margalit, E., Vaadia, E., 1993. Spatio- temporal firing patterns in frontal cortex of behaving monkeys. J. Neurophysiol. 70, 1629-1643.
  3. Ajjanagadde, V., Shastri, L., 1991. Rules and variables in neural nets. Neural Comput. 3, 121-134.
  4. Alon, N., Dewdney, A., Ott, T.J., 1991. Efficient simulation of finite automata by neural nets. J. ACM 38 (2), 495-514.
  5. Alquézar, R., Sanfeliu, A., 1996. An algebraic framework to represent finite state machines in single-layer recurrent networks. Neural Comput. 7 (5), 931-949.
  6. Arbib, M.A., Billard, A., Iacoboni, M., Oztop, E., 2000. Synthetic brain imaging: grasping, mirror neurons and imitation. Neural Networks 13 (8/9), 975-997.
  7. Arbib, M.A., Fagg, A.H., Grafton, S.T., 2002. Synthetic PET imaging for grasping: from primate neurophysiology to human behavior. In: Sommer, F.T., Wichert, A. (Eds.), Exploratory Analysis and Data Modelling in Functional Neuroimaging. MIT Press, Boston, MA. Arbib, M.A., 2003. The Handbook of Brain Theory and Neural Net- works, 2nd ed. MIT Press, Cambridge, MA.
  8. Bienenstock, E., 1996. Composition. In: Aertsen, A., Braitenberg, V. (Eds.), Brain Theory. Biological Basis and Computational Princi- ples. Elsevier, Amsterdam.
  9. Billard, A., Hayes, G., 1999. DRAMA, a connectionist architecture for control and learning in autonomous robots. Adapt. Behav. J. 7 (1), 35-64.
  10. Braitenberg, V., 1978. Cell assemblies in the cerebral cortex. In: Hein, R., Palm, G. (Eds.), Lecture Notes in Biomathematics (21). Theoretical Approaches to Complex Systems. Springer, Berlin/Heidelberg/New York.
  11. Braitenberg, V., 1977. On the Texture of Brains. Springer, Berlin/Heidelberg/New York.
  12. Braitenberg, V., Schüz, A., 1991. Anatomy of the Cortex. Statistics and Geometry. Springer-Verlag, Berlin.
  13. Brooks, R.A., Stein, L.A., 1994. Building brains for bodies. Auton. Robot. 1, 7-25.
  14. Dayan, P., Abbot, L.F., 2001. Theoretical Neuroscience. MIT Press, Cambridge, MA.
  15. Diesmann, M., Gewaltig, M.O., Aertsen, A., 1999. Stable propagation of synchronous spiking in cortical neural networks. Nature 402 (6761), 529-533.
  16. Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M., Reitboeck, H.J., 1988. Coherent oscillations: a mechanism of feature linking in the visual cortex? Biol. Cybern. 60, 121-130.
  17. Elman, J.L., 1990. Finding structure in time. Cognitive Sci. 14, 179-211.
  18. Fay, R., Kaufmann, U., Knoblauch, A., Markert, H., Palm, G., 2004. Combining visual attention, object recognition and associative information processing in a neurobotic system. In: Wermter, S., Palm, G., Elshaw, M. (Eds.), Biomimetic Neural Learning for Intelligent Robots. Springer, Berlin/Heidelberg/New York.
  19. Fodor, J.A., Pylyshyn, Z.W., 1988. Connectionism and cognitive archi- tecture: a critical analysis. Cognition 28, 3-71.
  20. Frasconi, P., Gori, M., Maggini, M., Soda, G., 1996. Representation of finite state automata in recurrent radial basis function networks. Mach. Learn. 23 (1), 5-32.
  21. Giles, L., Gori, M., 1998. Adaptive Processing of Sequences and Data Structures. Springer, Heidelberg/New York.
  22. Giles, C.L., Omlin, Ch.W., 1993. Extraction, insertion and refinement of symbolic rules in dynamically driven recurrent networks. Con- nect. Sci. 5, 307-337.
  23. Goonatikale, S., Khebbal, S., 1995. Intelligent Hybrid Systems. John Wiley and Sons Ltd.
  24. Gori, M., Maggini, M., Soda, G., 1995. Learning Regular Grammars From Noisy Examples Using Recurrent Neural Networks Techni- cal Report. Dipartimento di Sistemi e Informatica, Universita di Firenze.
  25. Gori, M., Küchler, A., Sperduti, A., 1999. On the implementation of frontier-to-root tree automata in recursive neural networks. IEEE Trans. Neural Networks 10 (6), 1305-1314.
  26. Grossberg, S., Pessoa, L., 1998. Texture segregation, surface represen- tation, and figure-ground separation. Vision Res. 38, 2657-2684.
  27. Harnad, S., 1990. The symbol grounding problem. Physica D 42, 335-346.
  28. Hebb, D.O., 1949. The Organization of Behavior. A Neuropsycholog- ical Theory. Wiley, New York.
  29. Hopcroft, J., Ullman, J., 1969. Formal Languages and their Relation to Automata. Addison-Wesley, Reading, MA.
  30. Horne, B.G., Hush, D.R., 1996. Bounds on the complexity of recurrent neural network implementations of finite state machines. Neural Networks 9 (2), 243-252.
  31. Horwitz, B., Friston, K.J., Taylor, J.G., 2000. Neural modeling and functional brain imaging: an overview. Neural Networks 13, 829-846.
  32. Ikegaya, Y., Aaron, G., Cossart, R., Aronov, D., Lampl, I., Ferster, D., Yuste, R., 2004. Synfire chains and cortical songs: temporal modules of cortical activity. Science 304, 559-564.
  33. Knoblauch, A., 2003. Synchronization and Pattern Separation in Spik- ing Associative Memory and Visual Cortical Areas. PhD Thesis. Department of Neural Information Processing, University of Ulm, Germany.
  34. Knoblauch, A., Palm, G., 2001. Pattern separation and synchronization in spiking associative memories and visual areas. Neural Networks 14, 763-780.
  35. Knoblauch, A., Palm, G., 2002. Scene segmentation by spike syn- chronization in reciprocally connected visual areas. II: Global assemblies and synchronization on larger space and time scales. Biol. Cybern. 87 (3), 168-184.
  36. Knoblauch, A., Fay, R., Kaufmann, U., Markert, H., Palm, G., 2004. Associating words to visually recognized objects. In: Coradeschi, S., Saffiotti, A. (Eds.), Anchoring Symbols to Sensor Data. Papers from the AAAI Workshop. Technical Report WS-04-03. AAAI Press, Menlo Park, CA.
  37. Knoblauch, A., Markert, H., Palm, G., 2005. An associative corti- cal model of language understanding and action planning. In: Mira, J., Alvarez, J.R. (Eds.), Artificial Intelligence and Knowl- edge Engineering Applications: A Bioinspired Approach. Springer, Berlin/Heidelberg/New York.
  38. Krichmar, J.L., Edelman, G., 2002. Machine psychology: autonomous behavior, perceptual categorization and conditioning in a brain- based device. Cereb. Cortex 12 (8), 818-830.
  39. Lapicque, L., 1907. Recherches quantitatives sur l'excitation electrique des nerfs traitee comme une polarization. J. Physiol. Pathol. Gen. 9, 620-635.
  40. Lin, L., Osan, R., Tsien, J.Z., 2006. Organizing principles of real-time memory encoding: neural clique assemblies and universal neural codes. Trends Neurosci. 29 (1), 48-57.
  41. Maass, W., Sontag, E., 1999. Analog neural nets with gaussian or other common noise distribution cannot recognize arbitrary regular languages. Neural Comput. 11, 771-782.
  42. Markert, H., Knoblauch, A., Palm, G., 2005a. Associative language processing in cortical areas. In: Proceedings of the IEEE SMC UK-RI Chapter Conference 2005 on Applied Cybernetics.
  43. Markert, H., Knoblauch, A., Palm, G., 2005b. Detecting sequences and understanding language with neural associative memories and cell assemblies. In: Wermter, S., Palm, G., Elshaw, M. (Eds.), Biomimetic Neural Learning for Intelligent Robots. Springer, Hei- delberg/New York.
  44. Marr, D., 1982. Vision. W.H. Freeman, New York.
  45. McKinstry, J.L., Edelman, G.M., Kirchmar, J.L., 2006. A cerebellar model for predictive motor control tested in a brain-based device. Proc. Natl. Acad. Sci. U.S.A. 103, 3387-3392.
  46. Miikkulainen, R., Bijwaard, D., 1994. Parsing embedded clauses with distributed neural networks. In: Proceedings of the 12th National Conference on AI (AAAI'94). AAAI Press/MIT Press, Menlo Park, CA.
  47. Nilsson, N.J., 1998. Artificial Intelligence: A New Synthesis. Morgan Kaufmann, San Francisco.
  48. O'Reilly, R.C., Braver, T.S., Cohen, J.D., 1999. A biologically based computational model of working memory. In: Miyake, A., Shah, P. (Eds.), Models of Working Memory: Mechanisms of Active Main- tenance and Executive Control. Cambridge University Press, New York.
  49. Palm, G., 1980. On associative memories. Biol. Cybern. 36, 19- 31. Palm, G., 1982a. Neural Assemblies. An Alternative Approach to Arti- ficial Intelligence. Springer, Berlin.
  50. Palm, G., 1982b. Rules for synaptic changes and their relevance for the storage of information in the brain. Cybern. Syst. Res..
  51. Palm, G., 1986. Associative networks and cell assemblies. In: Palm, G., Aertsen, A. (Eds.), Brain Theory. Springer, Berlin/Heidelberg.
  52. Palm, G., 1987. Computing with neural networks. Science 235, 1227-1228.
  53. Palm, G., 1990a. Cell assemblies as a guideline for brain research. Concepts Neurosci. 1, 133-148.
  54. Palm, G., 1990. Local rules for synaptic modification in neural net- works. J. Comput. Neurosci.
  55. Palm, G., 1991. Memory capacities of local rules for synaptic modification. A comparative review. Concepts Neurosci. 2, 97- 128. Palm, G., 1992. On the information storage capacity of local learning. Neural Comput. 4, 703-711.
  56. Palm, G., Sommer, F.T., 1992. Information capacity in recurrent McCulloch-Pitts networks with sparsely coded memory states. Network 3, 177-186.
  57. Pulvermüller, F., 1999. Words in the brain's language. Behav. Brain Sci. 22, 253-336.
  58. Pulvermüller, F., 2002. The Neuroscience of Language: On Brain Circuits of Words and Serial Order. Cambridge University Press, Cambridge, UK.
  59. Pulvermüller, F., 2003. Sequence detectors as a basis of grammar in the brain. Theory Biosci. 122, 87-103.
  60. Rabiner, L., 1989. A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77, 257- 285.
  61. Rabiner, L., Juang, B.H., 1993. Fundamentals of Speech Recognition. Prentice Hall, Englewood Cliffs, NJ.
  62. Roy, D., 2002. Learning visually grounded words and syntax for a scene description task. Comput. Speech Lang. 16 (3), 353- 385.
  63. Rumelhart, D., McClelland, J., 1986. Parallel Distributed Processing. MIT Press, Cambridge, MA.
  64. Shastri, L., Ajjanagadde, V., 1993. From simple associations to system- atic reasoning: a connectionistic representation of rules, variables and dynamic bindings. Behav. Brain Sci. 16 (3), 417-494.
  65. Siegelmann, H.T., Giles, C.L., 1997. The complexity of language recognition by neural networks. Neurocomputing 15, 327- 345.
  66. Singer, W., Gray, C.M., 1995. Visual feature integration and the temporal correlation hypothesis. Annu. Rev. Neurosci. 18, 555- 586.
  67. Somers, D.C., Nelson, S.B., Sur, M., 1995. An emergent model of ori- entation selectivity in cat visual cortical simple cells. J. Neurosci. 15, 5448-5465.
  68. Steels, L., Brooks, R., 1995. The Artificial Life Route to Artificial Intel- ligence: Building Embodied, Situated Agents. Erlbaum, Hillsdale, NJ.
  69. Sun, R., Bookman, L.A., 1995. Computational Architectures Inte- grating Neural and Symbolic Processing. Kluwer Academic Publishers.
  70. Talmy, L., 2000. Toward a Cognitive Semantics. vol. I: Conceptual Structuring Systems. MIT Press, Cambridge, MA. von der Malsburg, C., 1986. Am I thinking assemblies? In: Palm, G., Aertsen, A. (Eds.), Brain Theory. Springer, Berlin/ Heidelberg.
  71. Wermter, S., 1995. Hybrid Connectionist Natural Language Process- ing. Chapman and Hall, London.
  72. Willshaw, D.J., Buneman, O.P., Longuet-Higgins, H.C., 1969. Non- holographic associative memory. Nature 222, 960-962.

Related papers

Semantic processing of English sentences using statistical computation based on neurophysiological models
Marcia T Mitchell

Computer programs that can accurately interpret natural human language and carry out instructions would improve the lives of people with language processing deficits and greatly benefit society in general. John von Neumann in theorized that the human brain utilizes its own unique statistical neuronal computation to decode language and that this produces specific patterns of neuronal activity. This paper extends Von Neumann’s theory to the processing of partial semantics of declarative sentences. I developed semantic neuronal network models that emulate key features of cortical language processing and accurately compute partial semantics of English sentences. The method of computation implements the MAYA Semantic Technique, a mathematical technique I previously developed to determine partial semantics of sentences within a natural language processing program. Here I further simplified the technique by grouping repeating patterns into fewer categories. Unlike other natural language programs, my approach computes three partial semantics. The results of this research show that the computation of partial semantics of a sentence uses both feedforward and feedback projection which suggest that the partial semantic presented in this research might be a conscious activity within the human brain.

View PDFchevron_right
Can Natural Language Processors Help Unlock the Black Box of Language Comprehension
Tyree Cowell

2018

One of the most overwhelming challenges facing researchers in Cognitive Science today is the mystery of how the brain parses and comprehends language. Studies in Computer Science are plagued by a similar challenge: how to create a program that can effectively process the grammar of human languages. Through this thesis, I hope to combine these two problems by comparing certain outputs of Google's SyntaxNet to EEG data of subjects listening to naturalistic language data, more specifically, Alice in Wonderland text. The Natural Language Processor I will be analyzing (SyntaxNet) processes each sentence incrementally. At each new state of the parser, it decides to perform one of the following actions: make a right or left arc between grammatically associated words, or shift another word onto the stack. Through some manipulation of the SyntaxNet code, as well as an original program, I extracted the timing and frequency of these decisions, pairing them with each word they were based on. Similarly, the EEG data collected by Dr. Brennan's Lab at the University of Michigan couples brain signals with the time at which a subject heard a specific word. Cross-referencing these two data sets allowed us to test the hypothesis that the parser's actions correspond, in some way, to cognitive actions that humans use to understand sentences. Our results showed that there is a correlation between the changing stack depth of the parser and positive frontal activation in the brain. There is also a correlation between number of right arcs and late negative activation. While these results have a number of possible causes, they support the hypothesis that natural language parsers may be useful in providing a plausible model for the black box of language comprehension in the human brain. USING NLP TO UNDERSTAND LANGUAGE IN THE BRAIN 3 Can Natural Language Processors Help Unlock the Black Box of Language Comprehension? Language processing in the brain has often been referred to as a "black box problem". The term was first coined by Noam Chomsky, the father of modern linguistics, in reference to the process of language acquisition and the "insufficient stimuli" conundrum. Linguists realized that while each child receives vastly different language input, they all form essentially the same internal grammars. Additionally, as language research progressed, researchers came to realize that there was no obvious way to observe the language processes that occur in the brain. In both of these situations, the input and output of language processing are accessible, but the internal mechanisms and processes are nearly impossible to observe. This is why the term "black box" is so accurate. Scientists have spent years attempting to get a glimpse into this black box, creating many innovating techniques in the process. Recent work has made progress on this problem by pairing natural language processors with behavioral and neuroscientific data. According to Hale et. al. (under review), this methodology has been applied to several NLP program so far, including Long Short Term Memory (LSTM) networks, which did not perform very well, and Recurrent Neural Network Grammar (RNNG) models, which performed decently well. (Here performance is measured by the existence of correlations between the NLP and neuroscientific data). The current study is a subset of this overarching investigation with a focus on Google's natural language processor, SyntaxNet, and its parser, Parsey McParseface. Due to the previously mentioned success with RNNG models, we suspect that SyntaxNet will correlate reasonably well with EEG data of participants listening to naturalistic language data. This is because, both RNNG models and SyntaxNet operate on the overall structure of building a phrase structure tree using a symbolic

View PDFchevron_right
Networks for memory, perception, and decision-making, and beyond to how the syntax for language might be implemented in the brain
Edmund Rolls

Brain research, 2014

Neural principles that provide a foundation for memory, perception, and decision-making include place coding with sparse distributed representations, associative synaptic modification, and attractor networks in which the storage capacity is in the order of the number of associatively modifiable recurrent synapses on any one neuron. Based on those and further principles of cortical computation, hypotheses are explored in which syntax is encoded in the cortex using sparse distributed place coding. Each cortical module 2-3mm in diameter is proposed to be formed of a local attractor neuronal network with a capacity in the order of 10,000 words (e.g. subjects, verbs or objects depending on the module). Such a system may form a deep language-of-thought layer. For the information to be communicated to other people, the modules in which the neurons are firing which encode the syntactic role, as well as which neurons are firing to specify the words, must be communicated. It is proposed that ...

View PDFchevron_right
Neural bases of syntax-semantics interface processing.
Sharlene D. Newman, Evguenia Malaia

Your article is protected by copyright and all rights are held exclusively by Springer Science +Business Media Dordrecht. This e-offprint is for personal use only and shall not be selfarchived in electronic repositories. If you wish to self-archive your article, please use the accepted manuscript version for posting on your own website. You may further deposit the accepted manuscript version in any repository, provided it is only made publicly available 12 months after official publication or later and provided acknowledgement is given to the original source of publication and a link is inserted to the published article on Springer's website. The link must be accompanied by the following text: "The final publication is available at link.springer.com".

View PDFchevron_right
Syntax meets semantics during brain logical computations
Andrew Fingelkurts

Progress in Biophysics & Molecular Biology, 2018

The discrepancy between syntax and semantics is a painstaking issue that hinders a better comprehension of the underlying neuronal processes in the human brain. In order to tackle the issue, we at first describe a striking correlation between Wittgenstein's Tractatus, that assesses the syntactic relationships between language and world, and Perlovsky's joint language-cognitive computational model, that assesses the semantic relationships between emotions and "knowledge instinct". Once we have established a correlation between a purely logical approach to the language and computable psychological activities, we aim to find the neural correlates of syntax and semantics in the human brain. Starting from topological arguments, we suggest that the semantic properties of a proposition are processed at a level that is higher dimensional than the functional level used by the brain for syntactical properties. In a fully reversible process, the syntactic elements embedded in Broca's area project into multiple scattered semantic cortical zones. The presence of higher functional dimensions gives rise to the increase in informational content that takes place in semantic expressions. Therefore, diverse features of human language and cognitive world can be assessed in terms of both the logic armor described by the Tractatus, and the neurocomputational techniques at hand. One of our motivations is to build a neurocomputational framework able to provide a feasible explanation for brain's semantic processing, in preparation for novel computers with nodes built into higher dimensions.

View PDFchevron_right

Related topics

  • Engineering
  • Bio-Inspired Systems
  • Associative Memory
  • Language understanding

    • Cited by

    Neural Information Processing in Cognition: We Start to Understand the Orchestra, but Where is the Conductor?
    Günther Palm

    Frontiers in Computational Neuroscience, 2016

    Research in neural information processing has been successful in the past, providing useful approaches both to practical problems in computer science and to computational models in neuroscience. Recent developments in the area of cognitive neuroscience present new challenges for a computational or theoretical understanding asking for neural information processing models that fulfill criteria or constraints from cognitive psychology, neuroscience and computational efficiency. The most important of these criteria for the evaluation of present and future contributions to this new emerging field are listed at the end of this article.

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved