Analysis of Tools for the Development of Conversational Agents

Proceedings of the 5th International Conference on Conversational User Interfaces

The aim of this study is to explore the challenges and experiences of conversational agent (CA) practitioners in order to highlight their practical needs and bring them into consideration within the scholarly sphere. A range of data scientists, conversational designers, executive managers and researchers shared their opinions and experiences through semi-structured interviews. They were asked about emerging trends, the challenges they face, and the design processes they follow when creating CAs. In terms of trends, fndings included mixed feelings regarding no-code solutions and a desire for a separation of roles. The challenges mentioned included a lack of socio-technical tools and conversational archetypes. Finally, practitioners followed diferent design processes and did not use the design processes described in the academic literature. These fndings were analyzed to establish links between practitioners' insights and discussions in related literature. The goal of this analysis is to highlight research-practice gaps by synthesising fve practitioner needs that are not currently being met. By highlighting these research-practice gaps and foregrounding the challenges and experiences of CA practitioners, we can begin to understand the extent to which emerging literature is infuencing industrial settings and where more research is needed to better support CA practitioners in their work. CCS CONCEPTS • Human-centered computing → Empirical studies in HCI; Empirical studies in collaborative and social computing; • Computing methodologies → Artifcial intelligence.

2013

We show that an agent with fairly good social conversational abilities can be built based on a limited number of topics and dialogue strategies if it is tailored to its intended users through a high degree of user involvement during an iterative development process. The technology used is pattern matching of question-answer pairs, coupled with strategies to handle: followup questions, utterances not understood, abusive utterances, repetitive utterances, and initiation of new topics.

In this study, we investigate the effects of conversational agents on communication and interaction when used to assist participants in developing an online portfolio. Data from 52 participants were gathered and analyzed through questionnaires, written reflections, transcripts of student-agent interactions, and focus groups. Data revealed that participants communicated with the agents on issues ranging from portfolio development to popular culture. Although participants did not view the agents as particularly helpful in completing class activities, they did use them as social companions throughout the four-week study. Implications of the findings for future design and research include: (a) learner-developed conversational agents, (b) improved "intelligence" with which agents deliver content-based knowledge, and (c) further developed virtual characters that can meet users' humanistic and utilitarian expectations.

2011

In this paper, we present an agent-based dialog simulation technique for learning new dialog strategies and evaluate conversational agents. Using this technique the effort necessary to acquire data required to train the dialog model and then explore new dialog strategies is considerably reduced. A set of measures has also been defined to evaluate the dialog strategy that is automatically learned and compare different dialog corpora. We have applied this technique to explore the space of possible dialog strategies and evaluate the dialogs acquired for a conversational agent that collects monitored data from patients suffering from diabetes.

The present contribution investigates the construction of dialogue structure for the use in human-machine interaction especially for robotic systems and embodied conversational agents. We are going to present a methodology and findings of a pilot study for the design of task-specific dialogues. Specifically, we investigated effects of dialogue complexity on two levels: First, we examined the perception of the embodied conversational agent, and second, we studied participants' performance following HRI. To do so, we manipulated the agent's friendliness during a brief conversation with the user in a receptionist scenario. The paper presents an overview of the dialogue system, the process of dialogue construction, and initial evidence from an evaluation study with na¨ve users (N = 40). These users interacted with the system in a task-based dialogue in which they had to ask for the way in a building unknown to them. Afterwards participants filled in a questionnaire. Our findings...

… Agents. MIT Press, …, 1999

Embodied conversational agents (ECAs) are not just computer interfaces represented by way of human or animal bodies. And they are not just interfaces where those human or animal bodies are lifelike or believable in their actions and their reactions to human users.

2004

Abstract Conversational agents integrate computational linguistics techniques with the communication channel of the Web to interpret and respond to statements made by users in ordinary natural language. Web-based conversational agents deliver high-volumes of interactive text-based dialogs. Recent years have seen significant activity in enterprise-class conversational agents.

Techniques and Effective Practices

This chapter describes an eHealth human-like conversational agent called Maria embedded in the Web page of the Health Department of the Junta de Andalucía in Spain. Although this implementation is based on a strong theoretical background, a more practical approach has been preferred for the real-world case hereby described. Maria has been designed to perform several major tasks: she can arrange a doctor’s appointment, reply to queries pertaining to many varied subdomains, and navigate through the Web page. One of Maria’s most remarkable features is the successful application of advanced design and humanizing techniques which endow her with unusual skills and an enticing personality. Maria has been developed by Intelligent Dialogue Systems (Indisys) within a larger scale Web development project conducted by Sadiel SA.

Proceedings of the 2018 EMNLP Workshop SCAI: The 2nd International Workshop on Search-Oriented Conversational AI

In task-oriented conversational agents, more attention has been usually devoted to assessing task effectiveness, rather than to how the task is achieved. However, conversational agents are moving towards more complex and human-like interaction capabilities (e.g. the ability to use a formal/informal register, to show an empathetic behavior), for which standard evaluation methodologies may not suffice. In this paper, we provide a novel methodology to assess-in a completely controlled way-the impact on the quality of experience of agent's interaction strategies. The methodology is based on a within subject design, where two slightly different transcripts of the same interaction with a conversational agent are presented to the user. Through a series of pilot experiments we prove that this methodology allows fast and cheap experimentation/evaluation, focusing on aspects that are overlooked by current methods.

To my parents, Márcio Velmovitsky and Fabiana Elkind Velmovitsky, my sister, Vanessa Elkind Velmovitsky, and my grandparents David Elkind Schwartz and Belinha Saubel Elkind, for the advices and encouragements throughout my life, and for always supporting me in my decisions, no matter what. I would like to thank Professor Lucena for his expert advise and for the opportunity of learning with his experience and knowledge throughout the research process and during my Master degree. To Dr. Marx Leles Viana, for the patience and invaluable support during the making of this work, and to Professor Ruy Luiz Milidiú, for introducing me to the subject of chatbots and encouraging me to pursue further studies in the area. I would like to thanks all my colleagues for their wonderful collaboration. This project would have been imposible without the support of CAPES and PUC-Rio. Thank you.