Why are you Silent? - Towards Responsiveness in Chatbots

Behavior Research Methods, 2007

IRJET, 2021

Improvements in technology have led to digitalization. Chatbots have become significant when it comes to transformation driven by mobility. Chatbots are meant to have interaction with humans. The conversation might take place in terms of speech and text inputs. There are varieties of chatbots implemented based on the complexity of human questions. Chatbots are being implanted in a variety of areas like HealthCare, Marketing, Entertainment, Education and so on. Because of its massive usage, it becomes significant to look after the usability of chatbots. People using chatbots will look for improvised features like reduced response time, user experience, consistent reply, relevancy, accuracy and precision, Multilingual , personalization, increased efficiency. So, the main intention behind this paper is to provide insights regarding amelioration done in chatbots in various areas. The research paper provides in-depth information regarding limitations and heuristics followed to improve the usability of chatbots in various aspects. In addition to this, Performance metrics on which chatbots are evaluated are included.

INTERNATIONAL JOURNAL OF MANAGEMENT & INFORMATION TECHNOLOGY

Chatbots enable machines to emulate human conversation, and recent developments have resulted in many online systems for the public to use. Although a few studies have investigated how humans interact with such programs, we are not aware of any that have analyzed transcripts in depth. In this study, students interacted with two Web-based chatbots, Rose and Mitsuku, for five minutes and evaluated how well they thought the software emulated human conversation. We reviewed the transcripts and found that students used fairly simple language and made many text errors. There were no significant differences between the two systems in our experimental measures, but we found that Rose tended to change the topic more often and Mitsuku seemed more argumentative.

Frontiers in Psychology, 2019

Internet Science, 2019

Chatbots are emerging as interactive systems. However, we lack knowledge on how to classify chatbots and how such classification can be brought to bear in analysis of chatbot interaction design. In this workshop paper, we propose a typology of chatbots to support such classification and analysis. The typology dimensions address key characteristics that differentiate current chatbots: the duration of the user's relation with the chatbot (short-term and long-term), and the locus of control for user's interaction with the chatbot (userdriven and chatbot-driven). To explore the usefulness of the typology, we present four example chatbot purposes for which the typology may support analysis of high-level chatbot interaction design. Furthermore, we analyse a sample of 57 chatbots according to the typology dimensions. The relevance and application of the typology for developers and service providers are discussed.

Humanities & Social Sciences Reviews, 2024

Purpose of the study: The purpose of this study is to review and analyze current automated techniques for monitoring chatbot conversations in the field of Conversational Artificial Intelligence. It aims to highlight the challenges and limitations of these techniques and provide insights into various metrics used to measure chatbot performance, with the goal of enhancing it. Methodology: The study employs a comprehensive literature review of existing automated techniques for monitoring chatbot conversations. Then, focusing on stateof-the-art approaches, the study introduces a division into numerical metrics (performance statistics and user engagement) and linguistic metrics (conversation analysis). Within conversation analysis, which is crucial for improving chatbot responses and accurately recognizing user intentions, the study identifies and presents three leading methods. Main Findings: The paper highlights that, while current chatbot numerical conversation metrics allow for continuous monitoring and enhancement of chatbot performance, there is still room for improvement in the automated linguistic analysis of chatbot conversations. Furthermore, monitoring chatbot conversations in an automatic way in order to implement adequate corrective actions, is an essential task for refining chatbot efficiency through continuous learning and adaptation. Applications of the study: The findings of this study have practical applications for businesses employing chatbots. By understanding the potential of current automated monitoring techniques and addressing their limitations, commercial chatbot systems can be improved for the benefit of customer satisfaction. Novelty/Originality of the study: The paper provides readers with the novel knowledge necessary to understand key metrics used to measure chatbot conversations from both numerical and linguistic perspectives. It adds value by guiding readers on how monitoring numerical metrics helps analyze chatbot interactions and explains how the automated linguistic analysis of chatbot conversation content is utilized in leading approaches.

2006

End-to-end evaluations of more conversational dialogue systems with naive users have uncovered severe usability problems that, among other things, result in low task completion rates. First analyses suggest that these problems are related to the system’s dialogue management and turn-taking behavior. This paper starts with a presentation of experimental results, which shed some light on the effects of that behavior. Based on these findings, some criteria which lie orthogonal to dialogue quality are spelled out. As such, they nevertheless constitute an integral part of a more comprehensive view on dialogue felicity as a function of dialogue quality and efficiency. Since the work on spoken and multimodal dialogue systems presented and discussed herein is aimed at more conversational and adaptive systems, we also show that in certain dialogical situations it is important for such systems to align linguistically towards the users. After describing the corresponding empirical experiments ...

2019

Human interaction partners can deal with interruptions and then resume the interaction. This ability should be emulated by social agents. How fast interruptions are handled might influence the overall impression of an agent. In this paper, we present the results of a user study on how a human dialog partner perceives the behavior of a virtual agent handling verbal user interruptions with different reaction times. The study goes beyond typical perception experiments by preserving the real-time interaction experience. For the evaluation, we rely on a parametrizable parallelized computational model that represents dialog flow, overlap detection, conflict recognition, and conflict handling in real-time. The evaluation results show that the timing of the agent's interruption handling in interactive human-agent dialogues is related to different interpersonal attitudes.

International Journal for Research in Applied Science & Engineering Technology (IJRASET), 2022

A conversational agent (chatbot) is a piece of software that is able to communicate with humans using natural language. Modeling conversation is an important task in natural language processing and artificial intelligence (AI). Indeed, ever since the birth of AI, creating a good chatbot remains one of the field's hardest challenges. While chatbots can be used for various tasks, in general they have to understand users' utterances and provide responses that are relevant to the problem at hand. In the past, methods for constructing chatbot architectures have relied on handwritten rules and templates or simple statistical methods. With the rise of deep learning these models were quickly replaced by end-to-end trainable neural networks around 2015. More specifically, the recurrent encoder-decoder model [Cho et al., 2014] dominates the task of conversational modeling. This architecture was adapted from the neural machine translation domain, where it performs extremely well. Since then a multitude of variations and features were presented that augment the quality of the conversation that chatbots are capable of the next section of my paper focuses on adapting the very recent Tranformer [Vaswani et al., 2017] model to the chatbot domain, which is currently the state-of-the-art in neural machine translation. I first present my experiments with the vanilla model, using conversations extracted from the Cornell Movie-Dialog Corpus [Danescu-Niculescu-Mizil and Lee, 2011]. Secondly, I augment the model with some of my ideas regarding the issues of encoder-decoder architectures. More specifically, I feed additional features into the model like mood or persona together with the raw conversation data. Finally, I conduct a detailed analysis of how the vanilla model performs on conversational data by comparing it to previous chatbot models and how the additional features, affect the quality of the generated responses. I.

Computers in Human Behavior, 2015

This study analyzed how communication changes when people communicate with an intelligent agent as opposed to with another human. We compared 100 instant messaging conversations to 100 exchanges with the popular chatbot Cleverbot along seven dimensions: words per message, words per conversation, messages per conversation, word uniqueness, and use of profanity, shorthand, and emoticons. A MANOVA indicated that people communicated with the chatbot for longer durations (but with shorter messages) than they did with another human. Additionally, human–chatbot communication lacked much of the richness of vocabulary found in conversations among people, and exhibited greater profanity. These results suggest that while human language skills transfer easily to human–chatbot communication, there are notable differences in the content and quality of such conversations.