Max Louwerse

Tilburg University, Cognitive Science and Artificial Intelligence, Faculty Member

University of Memphis, Psychology, Faculty Member

Followers

181

Following

Public Views

InterestsView All (6)

Uploads

Papers by Max Louwerse

Toward a taxonomy of a set of discourse markers in dialog: A theoretical and computational linguistic account.

Discourse Processes, 2003

Discourse markers are verbal and non-verbal devices that mark transition points in communication.... more Discourse markers are verbal and non-verbal devices that mark transition points in communication. They presumably facilitate the construction of a mental representation of the events described by the discourse. A taxonomy of these relational markers is one important beginning in investigations of language use. While several taxonomies of coherence relations have been proposed for monolog, only a few have been proposed for dialog. This paper presents a taxonomy of between-turn coherence relations in dialog and discusses several issues that arise out of constructing such a taxonomy. A large number of discourse markers was sampled from the Santa Barbara Corpus of Spoken American English. Two judges substituted each type of these markers for all other markers. This extensive substitution test determined whether hyponymous, hypernymous and synonymous relations existed between the markers from this corpus of dialogs. Evidence is presented for clustering coherence relations into four categories: direction, polarity, acceptance and empathics.

Download

The Multimodal Nature of Embodied Conversational Agents

Proceedings of the Annual Meeting of the Cognitive Science Society, 2009

Embodied conversational agents (ECA's) have become ubiquitous in human-computer interaction appli... more Embodied conversational agents (ECA's) have become ubiquitous in human-computer interaction applications. Implementing humanlike multimodal behavior in these agents is difficult, because so little is known about the alignment of facial expression, eye gaze, gesture, speech and dialogue act. The current study used the data from an extensive study of human face-to-face multimodal communication for the development of a multimodal ECA, and tested to what extent multimodal behavior influenced the human-computer interaction. Results from a persona assessment questionnaire showed the presence of facial expressions, gesture and intonation had a positive effect on five assessment scales. Eye tracking results showed facial expressions played a primarily pragmatic role, whereas intonation played a primarily semantic role. Gestures played a pragmatic or semantic role, dependent on their level of specificity. These findings shed light on multimodal behavior within and between human and digital dialogue partners.

Download

Immersive virtual reality benefits for visuomotor training

Applied Cognitive Psychology, 2021

Measures of prefrontal functional near-infrared spectroscopy in visuomotor learning

Experimental Brain Research, Feb 2, 2021

Functional near-infrared spectroscopy (fNIRS) is a promising technique for non-invasively assessi... more Functional near-infrared spectroscopy (fNIRS) is a promising technique for non-invasively assessing cortical brain activity during learning. This technique is safe, portable, and, compared to other imaging techniques, relatively robust to head motion, ocular and muscular artifacts and environmental noise. Moreover, the spatial resolution of fNIRS is superior to electroencephalography (EEG), a more commonly applied technique for measuring brain activity non-invasively during learning. Outcomes from fNIRS measures during learning might therefore be both sensitive to learning and to feedback on learning, in a different way than EEG. However, few studies have examined fNIRS outcomes in learning and no study to date additionally examined the effects of feedback. To address this apparent gap in the literature, the current study examined prefrontal cortex activity measured through fNIRS during visuomotor learning and how this measure is affected by task feedback. Activity in the prefrontal cortex decreased over the course of learning while being unaffected by task feedback. The findings demonstrate that fNIRS in the prefrontal cortex is valuable for assessing visuomotor learning and that this measure is robust to task feedback. The current study highlights the potential of fNIRS in assessing learning even under different task feedback conditions.

Download

The Effect of Context on Humor: A Constraint-Based Model of Comprehending Verbal Jokes

Discourse Processes, Jan 29, 2010

Two experiments were conducted to assess the effects of various constraints on the processing of ... more Two experiments were conducted to assess the effects of various constraints on the processing of jokes. Participants read humorous jokes and nonhumorous alternatives of the jokes, which were presented in 3 conditions that manipulated discourse context (comedy, political, and control). In Experiment 1, participants rated the funniness of texts and provided some recall data. In Experiment 2, participant&amp;#x27;s eye movements were collected to examine the effects of the different contexts on the online processing of the texts. Results confirmed ...

Benefits of immersive collaborative learning in CAVE-based virtual reality

International journal of educational technology in higher education, Nov 16, 2020

How to make the learning of complex subjects engaging, motivating, and effective? The use of imme... more How to make the learning of complex subjects engaging, motivating, and effective? The use of immersive virtual reality offers exciting, yet largely unexplored solutions to this problem. Taking neuroanatomy as an example of a visually and spatially complex subject, the present study investigated whether academic learning using a state-ofthe-art Cave Automatic Virtual Environment (CAVE) yielded higher learning gains compared to conventional textbooks. The present study leveraged a combination of CAVE benefits including collaborative learning, rich spatial information, embodied interaction and gamification. Results indicated significantly higher learning gains after collaborative learning in the CAVE with large effect sizes compared to a textbook condition. Furthermore, low spatial ability learners benefitted most from the strong spatial cues provided by immersive virtual reality, effectively raising their performance to that of high spatial ability learners. The present study serves as a concrete example of the effective design and implementation of virtual reality in CAVE settings, demonstrating learning gains and thus opening opportunities to more pervasive use of immersive technologies for education. In addition, the study illustrates how immersive learning may provide novel scaffolds to increase performance in those who need it most.

Download

Multimodal Communication in Face-to-Face Computer-Mediated Conversations

Proceedings of the Annual Meeting of the Cognitive Science Society, 2007

Multimodal communication involves multiple communicative channels including speech, facial moveme... more Multimodal communication involves multiple communicative channels including speech, facial movement, and gesture. Relatively few studies on how various communicative modalities are aligned in natural, face-to-face communication exist. As part of a larger project, the current study investigates how discourse structure, speech features, eye gaze, and facial movements interrelate during a map coordination task. The study thereby sheds light on multimodal communication in humans and gives guidelines for the development of embodied conversational agents.

Download

Neurophysiological changes in visuomotor sequence learning provide insight in general learning processes: Measures of brain activity, skin conductance, heart rate and respiration

International Journal of Psychophysiology, May 1, 2020

This is a PDF file of an article that has undergone enhancements after acceptance, such as the ad... more This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Download

Modality switch effects emerge early and increase throughout conceptual processing: Evidence from ERPs

We tested whether conceptual processing is modality-specific by tracking the time course of the C... more We tested whether conceptual processing is modality-specific by tracking the time course of the Conceptual Modality Switch effect. Forty-six participants verified the relation between property words and concept words. The conceptual modality of consecutive trials was manipulated in order to produce an Auditory-to-visual switch condition, a Haptic-tovisual switch condition, and a Visual-to-visual, no-switch condition. Event-Related Potentials (ERPs) were time-locked to the onset of the first word (property) in the target trials so as to measure the effect online and to avoid a within-trial confound. A switch effect was found, characterized by more negative ERP amplitudes for modality switches than noswitches. It proved significant in four typical time windows from 160 to 750 milliseconds post word onset, with greater strength in posterior brain regions, and after 350 milliseconds. These results suggest that conceptual processing may be modality-specific in certain tasks, but also that the early stage of processing is relatively amodal.

Download

How fundamental is embodiment to language comprehension? Constraints on embodied cognition

Many studies have shown that language comprehension can activate embodied representations. Whethe... more Many studies have shown that language comprehension can activate embodied representations. Whether these studies have shown that embodiment is a necessary and sufficient condition for language comprehension, or whether embodiment can be a matter of depth of processing is a question that has not been answered. Three experiments examined how iconicity and semanticity affected reaction times. Following word pairs were presented simultaneously, one above the other in either an iconic (branch above root) or reverse-iconic (root above branch) presentation (Experiment 1 and 2) or next to each other (Experiment 3). Half of the materials had a strong semantic relation, the other half a weak semantic relation, measured by word associations. Following Zwaan and Yaxley's ( ) study, a semantic judgment task yielded differences in RT for iconicity and semanticity. However, a (shallow) lexical decision task yielded the same differences for semanticity but not for iconicity. Horizontal presentation of the items again yielded semanticity but not iconicity differences. Moreover, in all experiments medium to large effect sizes were obtained for semanticity, and negligible effect sizes for iconicity. These findings suggest that embodied information like iconicity only receives enough activation when language is processed deeply.

Download

Detecting Manipulated Second Language Reading Texts

Proceedings of the Annual Meeting of the Cognitive Science Society, 2006

An important debate in the field of second language learning concerns assumed differences between... more An important debate in the field of second language learning concerns assumed differences between authentic and simplified text. Texts are simplified through techniques such as increasing high frequency vocabulary, controlling for connective and abstract language, revising complex syntax, and increasing redundancy within text to reduce cognitive processing load (Simenson 1987). Crossley ( ) argued that moderate, shallow, textual changes can significantly affect discourse structures and have the potential to affect discourse processing. Crossley also demonstrated that teachers of English could reliably distinguish these two text-types. In this study, we build on this research by evaluating the use of the computational tool, Coh-Metrix as a means to replicate human ability to distinguish the two text types. For this analysis, we constructed a corpus from both authentic and manipulated second language reading texts..In total, 224 texts used for second language instruction were excerpted from 11 intermediate reading textbooks. The dataset was divided into a training set (n=113 texts) and a test set (n=111 texts). Based on Crossley (2006), we selected five predictors from Coh-Metrix: logical connectors, textual abstractness/ambiguity, syntactic complexity, lexical coreference, and word information. The final three predictors were selected to closely match those used by humans when successfully discriminating between authentic and manipulated texts .

Download

The Role of Feedback in Learning Form-Meaning Mappings

Proceedings of the Annual Meeting of the Cognitive Science Society, 2009

Word learning is a complex activity whose mechanisms are not fully understood. For instance, ther... more Word learning is a complex activity whose mechanisms are not fully understood. For instance, there is an ongoing debate whether feedback is fundamental to learning the meaning of words. Three experiments and a simulation aimed to investigate the importance and role of feedback in learning a six-word vocabulary of an artificial language. In the experiments, participants had to guess what object in a visual array of objects was referred to by an utterance composed of unknown words. The words referred to objects characteristics like color, size or shape. Results showed that feedback led to a superior learning rate. However, some learning did occur without feedback. A simulation further illustrated possible mechanisms responsible for learning with and without feedback. These data are consistent with the idea that the mechanism by which feedback particularly helps learning consists of reducing the amount of ambiguity in the environment.

Download

Dialog Act Classification Using N-Gram Algorithms

The Florida AI Research Society, 2006

Speech act classification remains one of the challenges in natural language processing. This pape... more Speech act classification remains one of the challenges in natural language processing. This paper evaluates a classification system that assigns one of twelve dialog acts to an utterance from the Map Task Corpus. The dialog act classification system chooses a dialog act based on n-grams from a training set. The system's performance is comparable to other classification systems, like those using support vector machines. Performance is high given the fact that the system only considers an utterance out of context and from written input only. Moreover, the system's performance is on par with human performance.

Download

Non-invasive neurophysiological measures of learning: A meta-analysis

Neuroscience & Biobehavioral Reviews, Apr 1, 2019

Download

Avoiding the language-as-a-fixed-effect fallacy : How to estimate outcomes of linear mixed models

Cognitive Science, 2014

Since the 1970s, researchers in psycholinguistics and the cognitive sciences have been aware of t... more Since the 1970s, researchers in psycholinguistics and the cognitive sciences have been aware of the language-as-fixedeffect fallacy, or the importance in statistical analyses to not only average across participants (F1) but also across items (F2). Originally, the language-as-fixed-effect fallacy was countered by proposing a combined measure (minF') calculated by participant (F1) and item (F2) analyses. The scientific community, however, reported separate participant and item (F1 and F2) regression analyses instead. More recently, researchers have started using linear mixed models, a more robust statistical methodology that considers both random participant and item factors together in the same analysis. There are various benefits to using mixed models, including being more robust to missing values and unequal cell sizes than other linear models, such as ANOVAs. Yet it is unclear how conservative or liberal mixed methods are in comparison to the traditional methods. Moreover, reanalyzing previously completed work with linear mixed models seems cumbersome. It is therefore desirable to understand the benefits of linear mixed models and to know under what conditions results that are significant for one model might beget significant results for other models, in order to estimate the outcome of a mixed effect model based on traditional F1, F2, and minF' analyses. The current paper demonstrates that it is possible, at least for the most simplistic model, for an F or p value from a linear mixed model to be estimated from the same values from more traditional analyses.

Download

Age Differences in Learning-Related Neurophysiological Changes

Journal of Psychophysiology, Jul 1, 2023

Research in young adults has demonstrated that neurophysiological measures are able to provide in... more Research in young adults has demonstrated that neurophysiological measures are able to provide insight into learning processes. However, to date, it remains unclear whether neurophysiological changes during learning in older adults are comparable to those in younger adults. The current study addressed this issue by exploring age differences in changes over time in a range of neurophysiological outcome measures collected during visuomotor sequence learning. Specifically, measures of electroencephalography (EEG), skin conductance, heart rate, heart rate variability, respiration rate, and eye-related measures, in addition to behavioral performance measures, were collected in younger (M age = 27.24 years) and older adults (M age = 58.06 years) during learning. Behavioral responses became more accurate over time in both age groups during visuomotor sequence learning. Yet, older adults needed more time in each trial to enhance the precision of their movement. Changes in EEG during learning demonstrated a stronger increase in theta power in older compared to younger adults and a decrease in gamma power in older adults while increasing slightly in younger adults. No such differences between the two age groups were found on other neurophysiological outcome measures, suggesting changes in brain activity during learning to be more sensitive to age differences than changes in peripheral physiology. Additionally, differences in which neurophysiological outcomes were associated with behavioral performance on the learning task were found between younger and older adults. This indicates that the neurophysiological underpinnings of learning may differ between younger and older adults. Therefore, the current findings highlight the importance of taking age into account when aiming to gain insight into behavioral performance through neurophysiology during learning.

Download

Verifying properties from different emotions produces switching costs : Evidence for coarse-grained language statistics and fine-grained perceptual simulation

Cognitive Science, 2013

We investigated whether emotions are activated during comprehension of emotion words. In the firs... more We investigated whether emotions are activated during comprehension of emotion words. In the first part of the study, an experiment was conducted in which participants read sentence pairs each describing an emotional state and then engaged in a judgment task. Sentences were paired to either match or mismatch in emotion (happy, sad, or angry). We predicted that the sentences that mismatch in emotion produced longer reaction times than those where the emotion was the same, and that shifts between negative emotions had less of an impact. In the second part of the study, we calculated the frequency of first-order co-occurrences of nouns and adjectives related to happy, sad, and angry emotional states. This analysis demonstrated emotion words are more often accompanied by similar emotion words. Match and mismatch of emotion explained RTs as did statistical linguistic frequencies of the words. The combination of these two studies contributes to a growing body of research that supports the importance of both symbolic and perceptual processing of emotion.

Download

Corneal reflections and skin contrast yield better memory of human and virtual faces

Cognitive Research: Principles and Implications

Virtual faces have been found to be rated less human-like and remembered worse than photographic ... more Virtual faces have been found to be rated less human-like and remembered worse than photographic images of humans. What it is in virtual faces that yields reduced memory has so far remained unclear. The current study investigated face memory in the context of virtual agent faces and human faces, real and manipulated, considering two factors of predicted influence, i.e., corneal reflections and skin contrast. Corneal reflections referred to the bright points in each eye that occur when the ambient light reflects from the surface of the cornea. Skin contrast referred to the degree to which skin surface is rough versus smooth. We conducted two memory experiments, one with high-quality virtual agent faces (Experiment 1) and the other with the photographs of human faces that were manipulated (Experiment 2). Experiment 1 showed better memory for virtual faces with increased corneal reflections and skin contrast (rougher rather than smoother skin). Experiment 2 replicated these findings, s...

Download

Training Machine Learning Models to Detect Group Differences in Neurophysiological Data using Recurrence Quantification Analysis based Features

Proceedings of the 14th International Conference on Agents and Artificial Intelligence, 2022

Physiological data have shown to be useful in tracking and differentiating cognitive processes in... more Physiological data have shown to be useful in tracking and differentiating cognitive processes in a variety of experimental tasks, such as numerical skills and arithmetic tasks. Numerical skills are critical because they are strong predictors of levels of ability in cognitive domains such as literacy, attention, and understanding contexts of risk and uncertainty. In this work, we examined frontal and parietal electroencephalogram signals recorded from 36 healthy participants performing a mental arithmetic task. From each signal, six RQA-based features (Recurrence Rate, Determinism, Laminarity, Entropy, Maximum Diagonal Line Length and, Average Diagonal Line Length) were extracted and used for classification purposes to discriminate between participants performing proficiently and participants performing poorly. The results showed that the three classifiers implemented provided an accuracy above 0.85 on 5-fold cross-validation, suggesting that such features are effective in detecting performance independently from the specific classifiers used. Compared to other successful methods, RQA-based features have the potential to provide insights into the nature of the physiological dynamics and the patterns that differentiate levels of proficiency in cognitive tasks. a

Download

Realism of the face lies in skin and eyes: Evidence from virtual and human agents

Computers in Human Behavior Reports, 2021

Abstract Despite advancements in computer graphics and artificial intelligence, it remains unclea... more Abstract Despite advancements in computer graphics and artificial intelligence, it remains unclear which aspects of intelligent virtual agents (IVAs) make them identifiable as human-like agents. In three experiments and a computational study, we investigated which specific facial features in static IVAs contribute to judging them human-like. In Experiment 1, participants were presented with facial images of state-of-the-art IVAs and humans and asked to rate these stimuli on human-likeness. The results showed that IVAs were judged less human-like compared to photographic images of humans, which led to the hypothesis that the discrepancy in human-likeness was driven by skin and eye reflectance. A follow-up computational analysis confirmed this hypothesis, showing that the faces of IVAs had smoother skin and a reduced number of corneal reflections than human faces. In Experiment 2, we validated these findings by systematically manipulating the appearance of skin and eyes in a set of human photographs, including both female and male faces as well as four different races. Participants indicated as quickly as possible whether the image depicted a real human face or not. The results showed that smoothening the skin and removing corneal reflections affected the perception of human-likeness when quick perceptual decisions needed to be made. Finally, in Experiment 3, we combined the images of IVA faces and those of humans, unaltered and altered, and asked participants to rate them on human-likeness. The results confirmed the causal role of both features for attributing human-likeness. Both skin and eye reflectance worked in tandem in driving judgements regarding the extent to which the face was perceived human-like in both IVAs and humans. These findings are of relevance to computer graphics artists and psychology researchers alike in drawing attention to those facial characteristics that increase realism in IVAs.

Max Louwerse

Uploads

Papers by Max Louwerse

Log In