Everyday conversation requires listeners to quickly recognize verbal actions, so-called speech ac... more Everyday conversation requires listeners to quickly recognize verbal actions, so-called speech acts, from the underspecified linguistic code and prepare a relevant response within the tight time constraints of turn-taking. The goal of this study was to determine the time-course of speech act recognition by investigating oscillatory EEG activity during comprehension of spoken dialog. Participants listened to short, spoken dialogs with target utterances that delivered three distinct speech acts (Answers, Declinations, Pre-offers). The targets were identical across conditions at lexico-syntactic and phonetic/prosodic levels but differed in the pragmatic interpretation of the speech act performed. Speech act comprehension was associated with reduced power in the alpha/beta bands just prior to Declination speech acts, relative to Answers and Pre-offers. In addition, we observed reduced power in the theta band during the beginning of Declinations, relative to Answers. Based on the role of alpha and beta desynchronization in anticipatory processes, the results are taken to indicate that anticipation plays a role in speech act recognition. Anticipation of speech acts could be critical for efficient turn-taking, allowing interactants to quickly recognize speech acts and respond within the tight time frame characteristic of conversation. The results show that anticipatory processes can be triggered by the characteristics of the interaction, including the speech act type.
In conversation, turn transitions between speakers often occur smoothly, usually within a time wi... more In conversation, turn transitions between speakers often occur smoothly, usually within a time window of a few hundred milliseconds. It has been argued, on the basis of a button-press experiment [De Ruiter, J. P., Mitterer, H., & Enfield, N. J. (2006). Projecting the end of a speaker's turn: A cognitive cornerstone of conversation. Language, 82(3):515-535], that participants in conversation rely mainly on lexico-syntactic information when timing and producing their turns, and that they do not need to make use of intonational cues to achieve smooth transitions and avoid overlaps. In contrast to this view, but in line with previous observational studies, our results from a dialogue task and a button-press task involving questions and answers indicate that the identification of the end of intonational phrases is necessary for smooth turn-taking. In both tasks, participants never responded to questions (i.e., gave an answer or pressed a button to indicate a turn end) at turn-internal points of syntactic completion in the absence of an intonational phrase boundary. Moreover, in the button-press task, they often pressed the button at the same point of syntactic completion when the final word of an intonational phrase was cross-spliced at that location. Furthermore, truncated stimuli ending in a syntactic completion point but lacking an intonational phrase boundary led to significantly delayed button presses. In light of these results, we argue that earlier claims that intonation is not necessary for correct turn-end projection are misguided, and that research on turn-taking should continue to consider intonation as a source of turn-end cues along with other linguistic and communicative phenomena.
Language, cognition and neuroscience, Mar 14, 2019
The present study extends neuro-imaging into conversation through studying dialogue comprehension... more The present study extends neuro-imaging into conversation through studying dialogue comprehension. Conversation entails rapid responses, with negative semiotics for delay. We explored how expectations about the valence of the forthcoming response develop during the silence before the response and whether negative responses have mainly cognitive or social-emotional consequences. EEG-participants listened to questions from a spontaneous spoken corpus, cross-spliced with short/long gaps and "yes"/"no" responses. Preceding contexts biased listeners to expect the eventual response, which was hypothesised to translate to expectations for a shorter or longer gap. "No" responses showed a trend towards an early positivity, suggesting socio-emotional consequences. Within the long gap, expecting a "yes" response led to an earlier negativity, as well as a trend towards stronger theta-oscillations, after 300 milliseconds. This suggests that listeners anticipate/predict "yes" responses to come earlier than "no" responses, showing strong sensitivities to timing, which presumably promote hastening the pace of verbal interaction.
Research on Language and Social Interaction, Dec 29, 2016
This article reviews the prospects for the cross-fertilization of conversation analytic (CA) and ... more This article reviews the prospects for the cross-fertilization of conversation analytic (CA) and neurocognitive studies of conversation, focusing on turn taking. Although conversation is the primary ecological niche for language use, relatively little brain research has focused on interactive language use, partly due to the challenges of using brain-imaging methods that are controlled enough to perform sound experiments but still reflect the rich and spontaneous nature of conversation. Recently, though, brain researchers have started to investigate conversational phenomena-for example, by using "overhearer" or controlled interaction paradigms. We review neuroimaging studies related to turn-taking and sequence organization, phenomena historically described by CA. These studies, for example, show early action recognition and immediate planning of responses midway during an incoming turn. The review discusses studies with an eye to a fruitful interchange between CA and neuroimaging research on conversation and an indication of how these disciplines can benefit from each other.
A striking puzzle about language use in everyday conversation is that turn-taking latencies are u... more A striking puzzle about language use in everyday conversation is that turn-taking latencies are usually very short, whereas planning language production takes much longer. This implies overlap between language comprehension and production processes, but the nature and extent of such overlap has never been studied directly. Combining an interactive quiz paradigm with EEG measurements in an innovative way, we show that production planning processes start as soon as possible, that is, within half a second after the answer to a question can be retrieved (up to several seconds before the end of the question). Localization of ERP data shows early activation even of brain areas related to late stages of production planning (e.g., syllabification). Finally, oscillation results suggest an attention switch from comprehension to production around the same time frame. This perspective from interactive language use throws new light on the performance characteristics that language competence involves.
Cognitive processes involved in turn-taking within Dutch question-answer sequences: An EEG-study
International Conference on Control and Automation, 2014
In experimental psycholinguistics, research has focused overwhelmingly on either language compreh... more In experimental psycholinguistics, research has focused overwhelmingly on either language comprehension or language production. However, the ultimate goal should be to identify the cognitive mechanisms that occur during the actual use of comprehension and production in everyday life, for example in conversation. Turn-taking (Sacks, Schegloff, & Jefferson, 1974) and action-sequencing (Schegloff, 2007) are essential mechanisms in everyday communication, but have not been studied much yet in the domain of experimental psycholinguistics (but see De Ruiter, Mitterer, & Enfield, 2006; Magyari & De Ruiter, 2012; Roberts, Francis, & Morgan, 2006). CA, on the other hand, has focused extensively on these mechanisms, via observing the behavior of conversationalists. To bring these two research domains closer together, in the present study, we aim to look at the cognitive processes that form the foundations for observable turn-taking and action-sequencing behavior in social interaction. We used electroencephalography (EEG), a method that measures electrical brain activity with a very good time resolution, to address this question. In EEG, experimental control is necessary to identify patterns of brain activity related to specific cognitive processes. Using an interactive quiz paradigm in Dutch, we investigated the processes underlying turn-taking in question-answer sequences, while still exerting enough experimental control over both the presented questions and expected answers. Participants were asked quiz questions by a research assistant while their EEG was measured. These questions were in fact pre-recorded, but participants believed they were asked live and they did receive live feedback from the assistant. We compared 'early questions' such as "Which character, also called 007, appears in the famous movies?" in which answer planning could start early in the question (at "007") with matched 'late questions' like "Which character from the famous movies, is also called 007?" in which answer planning could only start at the last word. Reaction times were longer for answers ("James Bond") to late than to early questions. Averaging the EEG signal to the critical word ("007" in the example) in both types of questions, we first see a small N400 effect (Kutas & Hillyard, 1980), a negative effect after about 400 milliseconds, which has been related to processing of unexpected words in a sentence. After this negativity, a large positive effect emerges which appears to be sustained until the response. We tentatively interpret this positivity to reflect a cognitive process associated with the response, such as the retrieval of the answer or the planning of production. These results thus show that in a question-answer sequence, speakers appear to start planning their response immediately when they have enough information to do so. Further results in the domain of brain oscillations and localizations of the effects in the brain will also be discussed. These results imply that cognitive processes might have a different timing than interactional processes. In some cases (as in our 'early questions'), the cognitive processes might be ready while interactionally the answer has to be postponed until the speaker finished speaking, possibly freeing up cognitive resources for a better timing of the response, for example. In other cases, cognitive processes might still be ongoing while the speaker has already finished her turn, for example when the question is difficult, calling for other interactional mechanisms such as the use of fillers. Future studies might employ the patterns of brain activation found to determine the presence and timing of cognitive processes within different types of action-sequences
In conversation, negative responses to invitations, requests, offers, and the like are more likel... more In conversation, negative responses to invitations, requests, offers, and the like are more likely to occur with a delay-conversation analysts talk of them as dispreferred. Here we examine the contrastive cognitive load 'yes' and 'no' responses make, either when relatively fast (300 ms after question offset) or delayed (1000 ms). Participants heard short dialogues contrasting in speed and valence of response while having their EEG recorded. We found that a fast 'no' evokes an N400-effect relative to a fast 'yes'; however, this contrast disappeared in the delayed responses. 'No' responses, however, elicited a late frontal positivity both if they were fast and if they were delayed. We interpret these results as follows: a fast 'no' evoked an N400 because an immediate response is expected to be positive-this effect disappears as the response time lengthens because now in ordinary conversation the probability of a 'no' has increased. However, regardless of the latency of response, a 'no' response is associated with a late positivity, since a negative response is always dispreferred. Together these results show that negative responses to social actions exact a higher cognitive load, but especially when least expected, in immediate response.
We know that speech planning in conversational turn-taking can happen in overlap with the previou... more We know that speech planning in conversational turn-taking can happen in overlap with the previous turn and research suggests that it starts as early as possible, that is, as soon as the gist of the previous turn becomes clear. The present study aimed to investigate whether planning proceeds all the way up to the last stage of articulatory preparation (i.e., putting the articulators in place for the first phoneme of the response) and what the timing of this process is. Participants answered pre-recorded quiz questions (being under the illusion that they were asked live), while their tongue movements were measured using ultrasound. Planning could start early for some quiz questions (i.e., midway during the question), but late for others (i.e., only at the end of the question). The results showed no evidence for a difference between tongue movements in these two types of questions for at least two seconds after planning could start in early-planning questions, suggesting that speech p...
Specialty section: This article was submitted to Language Sciences, a section of the journal Fron... more Specialty section: This article was submitted to Language Sciences, a section of the journal Frontiers in Psychology
Turn-end Estimation in Conversational Turn-taking: The Roles of Context and Prosody
Discourse Processes, 2021
ABSTRACT This study investigated the role of contextual and prosodic information in turn-end esti... more ABSTRACT This study investigated the role of contextual and prosodic information in turn-end estimation by means of a button-press task. We presented participants with turns extracted from a corpus of telephone calls visually (i.e., in transcribed form, word-by-word) and auditorily, and asked them to anticipate turn ends by pressing a button. The availability of the previous conversational context was generally helpful for turn-end estimation in short turns only, and more clearly so in the visual task than in the auditory task. To investigate the role of prosody, we examined whether participants in the auditory task pressed the button close to turn-medial points likely to constitute turn ends based on lexico-syntactic information alone. We observed that the vast majority of such button presses occurred in the presence of an intonational boundary rather than in its absence. These results are consistent with the view that prosodic cues in the proximity of turn ends play a relevant role in turn-end estimation.
Imagine you want to tell your friend an anecdote about a class mate. Neither of you know the pers... more Imagine you want to tell your friend an anecdote about a class mate. Neither of you know the person’s name. How can you convey who you are referring to? You could refer to features of the person (e.g., ‘the one with red hair’ or ‘the tall one’) or use more metaphorical terms (i.e., ‘the teacher’s favorite’ or ‘the gamer’). Of all the things you could say, in principle, how do you decide what to say? This communication problem cuts at the core of some of the key challenges in explaining the computational cognitive infrastructure of human communication (de Ruiter et al., 2010). We report some first results from a consortium-based approach aimed at tackling these problems empirically and theoretically. We use cognitive agent-based simulations to investigate which cognitive capacities are necessary and sufficient to achieve mutual understanding. We manipulate signal ambiguity (e.g., ‘the tall one’ may refer to just one or many class mates), knowledge asymmetry (e.g., your friend may not...
Uploads
Papers by Sara Bögels