Speech Reduction, Intensity, and F0 Shape are Cues to Turn-Taking

Interspeech 2017, 2017

We present the results of a series of machine learning experiments aimed at exploring the differences and similarities in the production of turn-taking cues in American English and Argentine Spanish. An analysis of prosodic features automatically extracted from 21 dyadic conversations (12 En, 9 Sp) revealed that, when signaling Holds, speakers of both languages tend to use roughly the same combination of cues, characterized by a sustained final intonation, a shorter duration of turn-final interpausal units, and a distinct voice quality. However, in speech preceding Smooth Switches or Backchannels, we observe the existence of the same set of prosodic turn-taking cues in both languages, although the ways in which these cues are combined together to form complex signals differ. Still, we find that these differences do not degrade below chance the performance of cross-linguistic systems for automatically detecting turn-taking signals. These results are relevant to the construction of multilingual spoken dialogue systems, which need to adapt not only their ASR modules but also the way prosodic turn-taking cues are synthesized and recognized.

Frontiers in psychology, 2015

The anticipation of a speaker's next turn is a key element of successful conversation. This can be achieved using a multitude of cues. In natural conversation, the most important cue for adults to anticipate the end of a turn (and therefore the beginning of the next turn) is the semantic and syntactic content. In addition, prosodic cues, such as intonation, or visual signals that occur before a speaker starts speaking (e.g., opening the mouth) help to identify the beginning and the end of a speaker's turn. Early in life, prosodic cues seem to be more important than in adulthood. For example, it was previously shown that 3-year-old children anticipated more turns in observed conversations when intonation was available compared with when not, and this beneficial effect was present neither in younger children nor in adults (Keitel et al., 2013). In the present study, we investigated this effect in greater detail. Videos of conversations between puppets with either normal or fla...

Journal of Phonetics, 2015

In conversation, turn transitions between speakers often occur smoothly, usually within a time window of a few hundred milliseconds. It has been argued, on the basis of a button-press experiment [De Ruiter, J. P., Mitterer, H., & Enfield, N. J. (2006). Projecting the end of a speaker's turn: A cognitive cornerstone of conversation. Language, 82(3):515-535], that participants in conversation rely mainly on lexico-syntactic information when timing and producing their turns, and that they do not need to make use of intonational cues to achieve smooth transitions and avoid overlaps. In contrast to this view, but in line with previous observational studies, our results from a dialogue task and a button-press task involving questions and answers indicate that the identification of the end of intonational phrases is necessary for smooth turn-taking. In both tasks, participants never responded to questions (i.e., gave an answer or pressed a button to indicate a turn end) at turn-internal points of syntactic completion in the absence of an intonational phrase boundary. Moreover, in the button-press task, they often pressed the button at the same point of syntactic completion when the final word of an intonational phrase was cross-spliced at that location. Furthermore, truncated stimuli ending in a syntactic completion point but lacking an intonational phrase boundary led to significantly delayed button presses. In light of these results, we argue that earlier claims that intonation is not necessary for correct turn-end projection are misguided, and that research on turn-taking should continue to consider intonation as a source of turn-end cues along with other linguistic and communicative phenomena.

The Phonetics Symposium, 2002

In free dialogue, the speaking time may be distributed among the speakers in various ways. These turn-taking dynamics probably reflect interactional settings. The timing of utterances and pauses has been studied as early as in the 1930's, whereas the long-term prosodic properties of interaction and turn-taking dynamics have received less attention. In this preliminary study, we explore and visualize fragments of Finnish informal dialogue with regard to turn-taking dynamics and such prosodic features as F0, speech rate, and creaky voice.

This thesis presents a series of exploratory studies of how participants engaged in everyday conversation utilise linguistic resources in managing entry to and exit from talk. For each study a data-set is constructed from audio recordings of telephone calls; this data-set is then analysed in terms of interactional organisation and linguistic design. Analysis of interactional aspects of the talk is conducted according to the principles of Conversation Analysis; analysis of linguistic design focuses on phonetic details, employing both impressionistic and acoustic techniques. Throughout the studies, an attempt is made to relate the linguistic (and particularly the phonetic) design of talk to its function in interaction.

This study investigates co-speech movements as a function of the conversational turn exchange type, the type of speech material at a turn exchange, and the interlocutor's role as speaker or listener. A novel interactive protocol that mixes conversation and (non-read) nursery rhymes works to elicit many speech turns and co-speech movements within dyadic speech interaction. To evaluate a large amount of data, we use the density of cospeech movement as a quantitative measure. Results indicate that both turn exchange type and participant role are associated with variation in movement density for head and brow co-speech movement. Brow and head movement becomes denser as speakers approach overlapping speech exchanges, indicating that speakers increase their movement density as an interruptive exchange is approached. Similarly, head movement generally increases after such overlapping exchanges. Lastly, listeners display a higher rate of co-speech movement than speakers, both at speech turns and remote from them. Brow and head movements generally behave similarly across speech material types, conversational roles, and turn exchange types. On the whole, the study demonstrates that the quantitative co-speech movement density measure advanced here is useful in the study of co-speech movement and turn-taking.

Interspeech 2017

Speakers in conversations may adapt their turn pitch relative to that of preceding turns to signal alignment with their interlocutor. However, the reference frame for pitch matching across turns is still unclear. Researchers studying pitch in the context of conversation have argued for an initializing approach, in which turn pitch must be judged relative to pitch in preceding turns. However, perceptual studies have indicated that listeners are able to reliably identify the location of pitch values within an individual speaker's range; that is, even without conversational context, they are able to normalize to speakers. This would imply that speakers might match normalized pitch instead of absolute pitch. Using a combined quantitative-qualitative approach, we investigate the relationship between pitch in adjacent turns in spontaneous German conversation. We use two different methods of evaluating pitch in adjacent turns, reflecting normalizing and initializing approaches respectively. We find that the results are well correlated with conversational participants' evaluation of the conversation. Furthermore, evaluating locations with matched or mismatched pitch can help distinguish between blind and face-to-face conversational situations, as well as identifying locations where specific discourse strategies (such as tag questions) have been deployed.

Proceedings of FONETIK, 2022

Understanding the turn-taking system in conversation entails not only knowledge about the linguisticstructural and phonetic before Potential Turn Boundaries (PTBs), but crucially, the precise location of the transition space as well. To investigate the time domain and the phonological domain, we compare production and perception of turn ends in two related languages: German and Swedish. For the first part, we extracted pitch values at seven time points before PTBs from spontaneous speech produced in two-party conversations. The aim was to investigate the possible presence of specific patterns of variations that lead to either speaker change, floor keeping or backchannels. As no such patterns have emerged, for the second part, eye-tracking will be used to investigate the exact time point at which the ending of a turn can be projected by a listener and which acoustic signals are important for this prediction.

2004

On the basis of two-speaker spontaneous conversations, it is shown that the distributions of both pauses and speech-overlaps of telephone and face-to-face dialogues have different statistical properties. Pauses in a face-to-face dialogue last up to 4 times longer than pauses in telephone conversations in functionally comparable conditions. There is a high correlation (0.88 or larger) between the average pause duration for the two speakers across face-to-face dialogues and telephone dialogues. The data provided form a first quantitative analysis of the complex turn-taking mechanism evidenced in the dialogues available in the 9-million-word Spoken Dutch Corpus.

Computer Speech & Language, 2011

As interactive voice response systems become more prevalent and provide increasingly more complex functionality, it becomes clear that the challenges facing such systems are not solely in their synthesis and recognition capabilities. Issues such as the coordination of turn exchanges between system and user also play an important role in system usability. In particular, both systems and users have difficulty determining when the other is taking or relinquishing the turn. In this paper, we seek to identify turn-taking cues correlated with human-human turn exchanges which are automatically computable. We compare the presence of potential prosodic, acoustic, and lexico-syntactic turn-yielding cues in prosodic phrases preceding turn changes (smooth switches) vs. turn retentions (holds) vs. backchannels in the Columbia Games Corpus, a large corpus of task-oriented dialogues, to determine which features reliably distinguish between these three. We identify seven turn-yielding cues, all of which can be extracted automatically, for future use in turn generation and recognition in interactive voice response (IVR) systems. Testing hypothesis that these turn-yielding cues are linearly correlated with the occurrence of turn-taking attempts, we further demonstrate that, the greater the number of turn-yielding cues that are present, the greater the likelihood that a turn change will occur. We also identify six cues that precede backchannels, which will also be useful for IVR backchannel generation and recognition; these cues correlate with backchannel occurrence in a quadratic manner. We find similar results for overlapping and for non-overlapping speech.