Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Related Work
Cognitive Data
Embedding Evaluation Method
Multiple Hypotheses Testing
Results & Discussion
Cognitive Data Implications
Correlation with Extrinsic Evaluation Results
Conclusion
References
All Topics
Computer Science
Artificial Intelligence

CogniVal: A Framework for Cognitive Word Embedding Evaluation

Profile image of Nicolas LangerNicolas Langer

2019, Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)

https://doi.org/10.18653/V1/K19-1050
visibility

…

description

16 pages

link

1 file

Abstract

An interesting method of evaluating word representations is by how much they reflect the semantic representations in the human brain. However, most, if not all, previous works only focus on small datasets and a single modality. In this paper, we present the first multimodal framework for evaluating English word representations based on cognitive lexical semantics. Six types of word embeddings are evaluated by fitting them to 15 datasets of eyetracking, EEG and fMRI signals recorded during language processing. To achieve a global score over all evaluation hypotheses, we apply statistical significance testing accounting for the multiple comparisons problem. This framework is easily extensible and available to include other intrinsic and extrinsic evaluation methods. We find strong correlations in the results between cognitive datasets, across recording modalities and to their performance on extrinsic NLP tasks.

Related papers

A Survey of Word Embeddings Evaluation Methods
Amir Bakarov

Word embeddings are real-valued word representations able to capture lexical semantics and trained on natural language corpora. Models proposing these representations have gained popularity in the recent years, but the issue of the most adequate evaluation method still remains open. This paper presents an extensive overview of the field of word embeddings evaluation, highlighting main problems and proposing a typology of approaches to evaluation, summarizing 16 intrinsic methods and 12 extrinsic methods. I describe both widely-used and experimental methods, systematize information about evaluation datasets and discuss some key challenges.

View PDFchevron_right
Intrinsic Evaluations of Word Embeddngs: What Can We Do Better?
Anna Rogers (Gladkova), Aleksandr Drozd

Proceedings of The 1st Workshop on Evaluating Vector Space Representations for NLP

This paper presents an analysis of existing methods for the intrinsic evaluation of word embeddings. We show that the main methodological premise of such evaluations is "interpretability" of word embeddings: a "good" embedding produces results that make sense in terms of traditional linguistic categories. This approach is not only of limited practical use, but also fails to do justice to the strengths of distributional meaning representations. We argue for a shift from abstract ratings of word embedding "quality" to exploration of their strengths and weaknesses.

View PDFchevron_right
Validating Word Embedding as a Tool for the Psychological Sciences
Victor Swift

2021

Word Embedding (WE) represents a class of machine learning techniques used to model the relative meaning of words, based on the contextual interchangeability of those words within text corpora. Because the measurement of meaning is fundamental to the psychological sciences, psychologists have begun to adapt this tool from computational linguistics to study the human mind. While WE has been validated as a means to recover the meanings contained in text, this tool has not been successfully validated as a means to recover the meanings contained in our heads. Accordingly, in this dissertation I seek to determine the extent to which WE is applicable to psychological research. 1 This is particularly true of WE algorithms such as fastText which are trained on subword information and can thereby approximate the meaning of words that were not present in the training corpora (Mikolov et al., 2017).

View PDFchevron_right
Improving Word Representations via Global Context and Multiple Word Prototypes
Christopher D Manning

Unsupervised word representations are very useful in NLP tasks both as inputs to learning algorithms and as extra word features in NLP systems. However, most of these models are built with only local context and one representation per word. This is problematic because words are often polysemous and global context can also provide useful information for learning word meanings. We present a new neural network architecture which 1) learns word embeddings that better capture the semantics of words by incorporating both local and global document context, and 2) accounts for homonymy and polysemy by learning multiple embeddings per word. We introduce a new dataset with human judgments on pairs of words in sentential context, and evaluate our model on it, showing that our model outperforms competitive baselines and other neural language models. 1

View PDFchevron_right
Evaluation of Word Vector Representations by Subspace Alignment
즈이 왠

Unsupervisedly learned word vectors have proven to provide exceptionally effective features in many NLP tasks. Most common intrinsic evaluations of vector quality measure correlation with similarity judgments. However, these often correlate poorly with how well the learned representations perform as features in downstream evaluation tasks. We present QVEC—a computation-ally inexpensive intrinsic evaluation measure of the quality of word embeddings based on alignment to a matrix of features extracted from manually crafted lexical resources—that obtains strong correlation with performance of the vectors in a battery of downstream semantic evaluation tasks. 1

View PDFchevron_right
Interpretable semantic vectors from a joint model of brain-and text-based meaning
Tom Mitchell

Vector space models (VSMs) represent word meanings as points in a high dimensional space. VSMs are typically created using a large text corpora, and so represent word semantics as observed in text. We present a new algorithm (JNNSE) that can incorporate a measure of semantics not previously used to create VSMs: brain activation data recorded while people read words. The resulting model takes advantage of the complementary strengths and weaknesses of corpus and brain activation data to give a more complete representation of semantics. Evaluations show that the model 1) matches a behavioral measure of semantics more closely, 2) can be used to predict corpus data for unseen words and 3) has predictive power that generalizes across brain imaging technologies and across subjects. We believe that the model is thus a more faithful representation of mental vocabularies.

View PDFchevron_right
A Survey On Neural Word Embeddings
Selma Tekir

ArXiv, 2021

Understanding human language has been a sub-challenge on the way of intelligent machines. The study of meaning in natural language processing (NLP) relies on the distributional hypothesis where language elements get meaning from the words that co-occur within contexts. The revolutionary idea of distributed representation for a concept is close to the working of a human mind in that the meaning of a word is spread across several neurons, and a loss of activation will only slightly affect the memory retrieval process. Neural word embeddings transformed the whole field of NLP by introducing substantial improvements in all NLP tasks. In this survey, we provide a comprehensive literature review on neural word embeddings. We give theoretical foundations and describe existing work by an interplay between word embeddings and language modeling. We provide broad coverage on neural word embeddings, including early word embeddings, embeddings targeting specific semantic relations, sense embeddi...

View PDFchevron_right
Evaluating Word Embeddings with Categorical Modularity
Silvia Casacuberta

2021

We introduce categorical modularity, a novel low-resource intrinsic metric to evaluate word embedding quality. Categorical modularity is a graph modularity metric based on the k-nearest neighbor graph constructed with embedding vectors of words from a fixed set of semantic categories, in which the goal is to measure the proportion of words that have nearest neighbors within the same categories. We use a core set of 500 words belonging to 59 neurobiologically motivated semantic categories in 29 languages and analyze three word embedding models per language (FastText, MUSE, and subs2vec). We find moderate to strong positive correlations between categorical modularity and performance on the monolingual tasks of sentiment analysis and word similarity calculation and on the cross-lingual task of bilingual lexicon induction both to and from English. Overall, we suggest that categorical modularity provides non-trivial predictive information about downstream task performance, with breakdown...

View PDFchevron_right
Circles are like Ellipses, or Ellipses are like Circles? Measuring the Degree of Asymmetry of Static and Contextual Word Embeddings and the Implications to Representation Learning
Sadhana Kumaravel

Proceedings of the ... AAAI Conference on Artificial Intelligence, 2021

Human judgments of word similarity have been a popular method of evaluating the quality of word embedding. But it fails to measure the geometry properties such as asymmetry. For example, it is more natural to say "Ellipses are like Circles" than "Circles are like Ellipses". Such asymmetry has been observed from the word evocation experiment, where one word is used to recall another. This association data have been understudied for measuring embedding quality. In this paper, we use three well-known evocation datasets for the purpose and study both static embedding as well as contextual embedding, such as BERT. To fight for the dynamic nature of BERT embedding, we probe BERT's conditional probabilities as a language model, using a large number of Wikipedia contexts to derive a theoretically justifiable Bayesian asymmetry score. The result shows that the asymmetry judgment and similarity judgments disagree, and asymmetry judgment aligns with its strong performance on "extrinsic evaluations". This is the first time we can show contextual embeddings's strength on intrinsic evaluation, and the asymmetry judgment provides a new perspective to evaluate contextual embedding and new insights for representation learning.

View PDFchevron_right
Evaluating Neural Word Representations in Tensor-Based Compositional Settings
Mehrnoosh Sadrzadeh

We provide a comparative study between neural word representations and traditional vector spaces based on co-occurrence counts, in a number of compositional tasks. We use three different semantic spaces and implement seven tensor-based compositional models, which we then test (together with simpler additive and multiplicative approaches) in tasks involving verb disambiguation and sentence similarity. To check their scalability, we additionally evaluate the spaces using simple compositional methods on larger-scale tasks with less constrained language: paraphrase detection and dialogue act tagging. In the more constrained tasks, co-occurrence vectors are competitive, although choice of compositional method is important; on the larger-scale tasks, they are outperformed by neural word embeddings, which show robust, stable performance across the tasks.

View PDFchevron_right

References (53)

  1. Samira Abnar, Rasyan Ahmed, Max Mijnheer, and Willem Zuidema. 2018. Experiential, distributional and dependency-based word embeddings have com- plementary roles in decoding brain activity. In Pro- ceedings of the 8th Workshop on Cognitive Modeling and Computational Linguistics (CMCL 2018), pages 57-66.
  2. Amir Bakarov. 2018a. Can eye movement data be used as ground truth for word embeddings evaluation? arXiv preprint arXiv:1804.08749.
  3. Amir Bakarov. 2018b. A survey of word em- beddings evaluation methods. arXiv preprint arXiv:1801.09536.
  4. Lisa Beinborn, Samira Abnar, and Rochelle Choenni. 2019. Robust evaluation of language-brain encoding experiments. arXiv preprint arXiv:1904.02547.
  5. Jonathan R Brennan, Edward P Stabler, Sarah E Van Wagenen, Wen-Ming Luh, and John T Hale. 2016. Abstract linguistic structure correlates with temporal activity during naturalistic comprehension. Brain and Language, 157:81-94.
  6. Michael P Broderick, Andrew J Anderson, Giovanni M Di Liberto, Michael J Crosse, and Edmund C Lalor. 2018. Electrophysiological correlates of semantic dissimilarity reflect the comprehension of natural, narrative speech. Current Biology, 28(5):803-809.
  7. Marco Catani, Derek K Jones, and Dominic H Ffytche. 2005. Perisylvian language networks of the human brain. Annals of Neurology: Official Journal of the American Neurological Association and the Child Neurology Society, 57(1):8-16.
  8. Billy Chiu, Anna Korhonen, and Sampo Pyysalo. 2016. Intrinsic evaluation of word vectors fails to predict extrinsic performance. In Proceedings of the 1st Workshop on Evaluating Vector-Space Representa- tions for NLP, pages 1-6.
  9. Uschi Cop, Nicolas Dirix, Denis Drieghe, and Wouter Duyck. 2017. Presenting GECO: An eyetracking corpus of monolingual and bilingual sentence read- ing. Behavior research methods, 49(2):602-615.
  10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume 1 (Long and Short Papers), pages 4171-4186.
  11. Rotem Dror, Gili Baumer, Marina Bogomolov, and Roi Reichart. 2017. Replicability analysis for natural language processing: Testing significance with mul- tiple datasets. Transactions of the Association for Computational Linguistics, 5:471-486.
  12. Rotem Dror, Gili Baumer, Segev Shlomov, and Roi Re- ichart. 2018. The hitchhiker's guide to testing statis- tical significance in natural language processing. In Proceedings of the 56th Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 1383-1392.
  13. Allyson Ettinger, Naomi Feldman, Philip Resnik, and Colin Phillips. 2016. Modeling N400 amplitude us- ing vector space models of word representation. In CogSci.
  14. Stefan L Frank. 2017. Word embedding distance does not predict word reading time.
  15. Stefan L Frank, Irene Fernandez Monsalve, Robin L Thompson, and Gabriella Vigliocco. 2013. Read- ing time data for evaluating broad-coverage models of english sentence processing. Behavior Research Methods, 45(4):1182-1190.
  16. Stefan L Frank, Leun J Otten, Giulia Galli, and Gabriella Vigliocco. 2015. The ERP response to the amount of information conveyed by words in sen- tences. Brain and language, 140:1-11.
  17. Stefan L Frank and Roel M Willems. 2017. Word predictability and semantic similarity show distinct patterns of brain activity during language compre- hension. Language, Cognition and Neuroscience, 32(9):1192-1203.
  18. Sahar Ghannay, Benoit Favre, Yannick Esteve, and Nathalie Camelin. 2016. Word embedding evalua- tion and combination. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), pages 300-305.
  19. Anna Gladkova and Aleksandr Drozd. 2016. Intrinsic evaluations of word embeddings: What can we do better? In Proceedings of the 1st Workshop on Eval- uating Vector-Space Representations for NLP, pages 36-42.
  20. Liberty S Hamilton and Alexander G Huth. 2018. The revolution will not be controlled: Natural stimuli in speech neuroscience. Language, Cognition and Neuroscience, pages 1-10.
  21. Olaf Hauk and Friedemann Pulvermüller. 2004. Ef- fects of word length and frequency on the human event-related potential. Clinical Neurophysiology, 115(5):1090-1103.
  22. Nora Hollenstein, Jonathan Rotsztejn, Marius Troen- dle, Andreas Pedroni, Ce Zhang, and Nicolas Langer. 2018. ZuCo, a simultaneous EEG and eye- tracking resource for natural sentence reading. Sci- entific Data.
  23. Nora Hollenstein and Ce Zhang. 2019. Entity recog- nition at first sight: Improving NER with eye move- ment information. In NAACL.
  24. Alexander G Huth, Wendy A de Heer, Thomas L Grif- fiths, Frédéric E Theunissen, and Jack L Gallant. 2016. Natural speech reveals the semantic maps that tile human cerebral cortex. Nature, 532(7600):453- 458.
  25. Marcel A Just and Patricia A Carpenter. 1980. A theory of reading: From eye fixations to comprehension. Psychological review, 87(4):329.
  26. Alan Kennedy, Robin Hill, and Joël Pynte. 2003. The Dundee corpus. In Proceedings of the 12th Euro- pean Conference on Eye Movement.
  27. AK Laurinavichyute, Irina A Sekerina, SV Alexeeva, and KA Bagdasaryan. 2017. Russian Sentence Cor- pus: Benchmark measures of eye movements in reading in Cyrillic.
  28. Alessandro Lopopolo, Stefan L Frank, Antal Van den Bosch, Annabel Nijhof, and Roel M Willems. 2018. The Narrative Brain Dataset (NBD), an fMRI dataset for the study of natural language processing in the brain. In LREC 2018 Workshop on Linguistic and Neuro-Cognitive Resources (LiNCR). LREC.
  29. Steven G Luke and Kiel Christianson. 2017. The Provo Corpus: A large eye-tracking corpus with predictability norms. Behavior Research Methods, pages 1-8.
  30. Rob van der Goot Malvina Nissim, Rik van Noord. 2019. Fair is better than sensational: Man is to doctor as woman is to doctor. arXiv preprint arXiv:1905.09866.
  31. Francis M Miezin, L Maccotta, JM Ollinger, SE Pe- tersen, and RL Buckner. 2000. Characterizing the hemodynamic response: effects of presentation rate, sampling procedure, and the possibility of ordering brain activity based on relative timing. Neuroimage, 11(6):735-759.
  32. Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. 2018. Ad- vances in pre-training distributed word representa- tions. In Proceedings of the International Confer- ence on Language Resources and Evaluation (LREC 2018).
  33. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- rado, and Jeff Dean. 2013. Distributed representa- tions of words and phrases and their compositional- ity. In Advances in neural information processing systems, pages 3111-3119.
  34. George A Miller and Christiane Fellbaum. 1992. Wordnet and the organization of lexical memory. In Intelligent tutoring systems for foreign language learning, pages 89-102. Springer.
  35. Abhijit Mishra, Diptesh Kanojia, and Pushpak Bhat- tacharyya. 2016. Predicting readers' sarcasm under- standability by modeling gaze behavior. In AAAI, pages 3747-3753.
  36. Abhijit Mishra, Diptesh Kanojia, Seema Nagar, Kuntal Dey, and Pushpak Bhattacharyya. 2017. Scanpath complexity: Modeling reading effort using gaze in- formation. In AAAI, pages 4429-4436.
  37. Tom M Mitchell, Svetlana V Shinkareva, Andrew Carl- son, Kai-Min Chang, Vicente L Malave, Robert A Mason, and Marcel Adam Just. 2008. Predicting human brain activity associated with the meanings of nouns. Science, 320(5880):1191-1195.
  38. Christoph Mulert. 2013. Simultaneous EEG and fMRI: towards the characterization of structure and dynam- ics of brain networks. Dialogues in clinical neuro- science, 15(3):381.
  39. Brian Murphy, Leila Wehbe, and Alona Fyshe. 2018. Decoding language from the brain. Language, cog- nition, and computational models, page 53.
  40. Neha Nayak, Gabor Angeli, and Christopher D Man- ning. 2016. Evaluating word embeddings using a representative suite of practical tasks. In Proceed- ings of the 1st Workshop on Evaluating Vector-Space Representations for NLP, pages 19-23.
  41. Jeffrey Pennington, Richard Socher, and Christo- pher D. Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Lan- guage Processing, pages 1532-1543.
  42. Francisco Pereira, Bin Lou, Brianna Pritchett, Samuel Ritter, Samuel J Gershman, Nancy Kanwisher, Matthew Botvinick, and Evelina Fedorenko. 2018. Toward a universal decoder of linguistic meaning from brain activation. Nature communications, 9(1):963.
  43. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word rep- resentations. In Proceedings of NAACL.
  44. Cathy J Price. 2012. A review and synthesis of the first 20 years of PET and fMRI studies of heard speech, spoken language and reading. Neuroimage, 62(2):816-847.
  45. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Nat- ural Language Processing, pages 2383-2392.
  46. Keith Rayner. 1998. Eye movements in reading and information processing: 20 years of research. Psy- chological bulletin, 124(3):372.
  47. Joao António Rodrigues, Ruben Branco, João Silva, Chakaveh Saedi, and António Branco. 2018. Pre- dicting brain activation with WordNet embeddings. In Proceedings of the Eight Workshop on Cognitive Aspects of Computational Language Learning and Processing, pages 1-5.
  48. Anna Rogers, Shashwath Hosur Ananthakrishna, and Anna Rumshisky. 2018. What's in your embedding, and how it predicts task performance. In Proceed- ings of the 27th International Conference on Com- putational Linguistics, pages 2690-2703.
  49. Chakaveh Saedi, António Branco, João António Ro- drigues, and João Silva. 2018. WordNet embed- dings. In Proceedings of The Third Workshop on Representation Learning for NLP, pages 122-131.
  50. Dan Schwartz and Tom Mitchell. 2019. Understanding language-elicited EEG data by predicting it from a fine-tuned language model. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, Volume 1 (Long and Short Papers), pages 43-57.
  51. Anders Søgaard. 2016. Evaluating word embeddings with fMRI and eye-tracking. In Proceedings of the 1st Workshop on Evaluating Vector-Space Represen- tations for NLP, pages 116-121.
  52. Erik F Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. In Proceedings of the 7th Conference on Natural Lan- guage Learning, volume 4, pages 142-147.
  53. Leila Wehbe, Ashish Vaswani, Kevin Knight, and Tom Mitchell. 2014. Aligning context-based statistical models of language with brain activity during read- ing. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 233-243.

Related papers

Evaluating word embeddings with fMRI and eye-tracking
Anders Søgaard

Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP, 2016

The workshop CfP assumes that downstream evaluation of word embeddings is impractical, and that a valid evaluation metric for pairs of word embeddings can be found. I argue below that if so, the only meaningful evaluation procedure is comparison with measures of human word processing in the wild. Such evaluation is non-trivial, but I present a practical procedure here, evaluating word embeddings as features in a multi-dimensional regression model predicting brain imaging or eyetracking word-level aggregate statistics.

View PDFchevron_right
Decoding Word Embeddings with Brain-Based Semantic Features
Emmanuele Chersoni

Computational Linguistics

Word embeddings are vectorial semantic representations built with either counting or predicting techniques aimed at capturing shades of meaning from word co-occurrences. Since their introduction, these representations have been criticized for lacking interpretable dimensions. This property of word embeddings limits our understanding of the semantic features they actually encode. Moreover, it contributes to the “black box” nature of the tasks in which they are used, since the reasons for word embedding performance often remain opaque to humans. In this contribution, we explore the semantic properties encoded in word embeddings by mapping them onto interpretable vectors, consisting of explicit and neurobiologically motivated semantic features (Binder et al. 2016). Our exploration takes into account different types of embeddings, including factorized count vectors and predict models (Skip-Gram, GloVe, etc.), as well as the most recent contextualized representations (i.e., ELMo and BERT...

View PDFchevron_right
How to evaluate word embeddings? On importance of data efficiency and simple supervised tasks
Stanislaw Jastrzebski

2017

Maybe the single most important goal of representation learning is making subsequent learning faster. Surprisingly, this fact is not well reflected in the way embeddings are evaluated. In addition, recent practice in word embeddings points towards importance of learning specialized representations. We argue that focus of word representation evaluation should reflect those trends and shift towards evaluating what useful information is easily accessible. Specifically, we propose that evaluation should focus on data efficiency and simple supervised tasks, where the amount of available data is varied and scores of a supervised model are reported for each subset (as commonly done in transfer learning). In order to illustrate significance of such analysis, a comprehensive evaluation of selected word embeddings is presented. Proposed approach yields a more complete picture and brings new insight into performance characteristics, for instance information about word similarity or analogy ten...

View PDFchevron_right
Experiential, Distributional and Dependency-based Word Embeddings have Complementary Roles in Decoding Brain Activity
Max Mijnheer

Proceedings of the 8th Workshop on Cognitive Modeling and Computational Linguistics (CMCL 2018)

View PDFchevron_right
Word Embedding Evaluation in Downstream Tasks and Semantic Analogies
Bernardo Consoli

2020

Language Models have long been a prolific area of study in the field of Natural Language Processing (NLP). One of the newer kinds of language models, and some of the most used, are Word Embeddings (WE). WE are vector space representations of a vocabulary learned by a non-supervised neural network based on the context in which words appear. WE have been widely used in downstream tasks in many areas of study in NLP. These areas usually use these vector models as a feature in the processing of textual data. This paper presents the evaluation of newly released WE models for the Portuguese langauage, trained with a corpus composed of 4.9 billion tokens. The first evaluation presented an intrinsic task in which WEs had to correctly build semantic and syntactic relations. The second evaluation presented an extrinsic task in which the WE models were used in two downstream tasks: Named Entity Recognition and Semantic Similarity between Sentences. Our results show that a diverse and comprehen...

View PDFchevron_right

Related topics

  • Computer Science
  • Artificial Intelligence
  • Natural Language Processing
  • Semantics (Computer Science)

    • Cited by

    Decoding EEG Brain Activity for Multi-Modal Natural Language Processing
    Nicolas Langer

    Frontiers in Human Neuroscience

    Until recently, human behavioral data from reading has mainly been of interest to researchers to understand human cognition. However, these human language processing signals can also be beneficial in machine learning-based natural language processing tasks. Using EEG brain activity for this purpose is largely unexplored as of yet. In this paper, we present the first large-scale study of systematically analyzing the potential of EEG brain activity data for improving natural language processing tasks, with a special focus on which features of the signal are most beneficial. We present a multi-modal machine learning architecture that learns jointly from textual input as well as from EEG features. We find that filtering the EEG signals into frequency bands is more beneficial than using the broadband signal. Moreover, for a range of word embedding types, EEG data improves binary and ternary sentiment classification and outperforms multiple baselines. For more complex tasks such as relati...

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved