Scripting Facial Expressions for X3D/VRML-based Embodied Agents

Proceedings of the 8th international conference on Intelligent user interfaces - IUI '03, 2003

Embodied agents present ongoing challenging agenda for research in multi-modal user interfaces and humancomputer-interaction. Such agent metaphors will only be widely applicable to online applications when there is a standardised way to map underlying engines with the visual presentation of the agents. This paper delineates the functions and specifications of a mark-up language for scripting the animation of virtual characters. The language is called: Character Mark-up Language (CML) and is an XML-based character attribute definition and animation scripting language designed to aid in the rapid incorporation of lifelike characters/agents into online applications or virtual reality worlds. This multi-modal scripting language is designed to be easily understandable by human animators and easily generated by a software process such as software agents. CML is constructed based jointly on motion and multi-modal capabilities of virtual life-like figures. The paper further illustrates the constructs of the language and describes a real-time execution architecture that demonstrates the use of such a language as a 4G language to easily utilise and integrate MPEG-4 media objects in online interfaces and virtual environments.

Cognitive Technologies, 2004

In this paper we propose a scripting language, called STEP, for embodied agents, in particular for their communicative acts like gestures and postures. Based on the formal semantics of dynamic logics, STEP has a solid semantic foundation, in spite of a rich number of variants of the compositional operators and interaction facilities on the worlds. STEP has been implemented in the distributed logic programming language DLP, a tool for the implementation of 3D web agents. In this paper, we also discuss principles of scripting language design for embodied agents and several aspects of the application of STEP.

2000

This paper presents a powerful animation engine, called Maxine, to develop applications with embodied animated agents. The engine, based on open source tools, allows the management of scenes and virtual characters, and allows multimodal and emotional interaction with the users. The virtual actors are provided with facial expressions, lyp-synch, emotional voice, and can vary their answers depending on their emotional

Applied Artificial Intelligence, 2010

Embodied agents are a powerful paradigm for current and future multimodal interfaces yet require high effort and expertise for their creation, assembly, and animation control. Therefore, open animation engines and high-level control languages are required to make embodied agents accessible to researchers and developers. We present EMBR, a new real-time character animation engine that offers a high degree of animation control via the EMBRScript language. We argue that a new layer of control, the animation layer, is necessary to keep the higher-level control layers (behavioral/functional) consistent and slim while allowing a unified and abstract access to the animation engine (e.g., for the procedural animation of nonverbal behavior). We also introduce new concepts for the high-level control of motion quality (spatial/temporal extent, power, fluidity). Finally, we describe the architecture of the EMBR engine, its integration into larger project contexts, and conclude with a concrete application.

2002

Lifelike animated agents present a challenging ongoing agenda for research. Such agent metaphors will only be widely applicable to on-line applications when there is a standardised way to map underlying engines with the visual presentation of the agents. This paper delineates functions and specifications of two markup languages for scripting the animation of virtual characters. These languages are: Character Markup Language (CML) which is an XML-based embodied agent character attribute definition and animation scripting language designed to aid in the rapid incorporation of lifelike agents into online applications or virtual reality worlds. CML is constructed based jointly on motion and multi-modal capabilities of virtual human figures. The other is Avatar Markup Language (AML) which is also an XML-based multimodal scripting language designed to be easily understandable by human animators and easily generated by a software process such as an agent. We illustrate the constructs of the language and delineate two real-time execution architectures for using these languages in online applications.

Computers & Graphics, 2008

This paper presents a powerful animation engine for developing applications with embodied animated agents called Maxine. The engine, based on open source tools, allows management of scenes and virtual characters, and pays special attention to multimodal and emotional interaction with the user. Virtual actors are endowed with facial expressions, lip-synch, emotional voice, and they can vary their answers depending on their own emotional state and the relationship with the user during conversation. Maxine virtual agents have been used in several applications: a virtual presenter was employed in MaxinePPT, a specific application developed to allow non-programmers to create 3D presentations easily using classical PowerPoint presentations; a virtual character was also used as an interactive interface to communicate with and control a domotic environment; finally, an interactive pedagogical agent was used to simplify and improve the teaching and practice of Computer Graphics subjects.

2002

In this paper we will outline the requirements for a software platform supporting embodied conversational agents. These requirements encompass computational concerns as well as presentation facilities, providing a suitably rich environment for applications deploying conversational agents. We will then propose a platform based on the distributed logic programming language DLP and X3D, the extensible Web3D format. Three case studies will be described, illustrating the potential of the DLP+X3D platform: a multiuser game with autonomous players, avatars commenting on rich media presentations, and a gesture scripting language for humanoids based on dynamic logic. In conclusion, we will discuss related work and review the evaluation criteria that pertain to the deployment of embodied conversational agents in rich media 3D environments.

Computers in Entertainment, 2009

In this article we describe our approach to generating convincing and empathetic facial animation. Our goal is to develop a robust facial animation platform that is usable and can be easily extended. We also want to facilitate the integration of research in the area and to directly incorporate the characters in interactive applications such as embodied conversational agents and games.

Center for Human …, 1993

Our overall goal is to produce as automatic as possible facial expressions with wrinkles from spoken input. We focus on two aspects of this problem: integration of the expressive wrinkles and generation of synchronized speechanimation. Our facial model integrates facial muscles deformations and bulges. We h a v e produced a high level programming language to automatically drive 3D animation of facial expressions from speech. Our system embodies rule-governed translation from speech and utterance meaning to facial expressions. We are concerned primarily with expressions conveying information correlated with the intonation of the voice, some of which are also correlated with a ect or emotion. We apply our automatic animation model to a new facial animation system which i n tegrates e ects of the facial motion as expressive wrinkles and muscles dependencies. We obtain then with this subtle criteria of modeling and motion an animation much more expressive and natural.

1999

In this paper we describe methods for the generation of real-time facial animation for various virtual actor using high level-actions. These high-level actions allow the user to forget the technical side of the animation, and focus only on the more abstract, more natural, and intuitive part of the facial animation. The mechanisms developed to generate interactive high-level real-time animation are described and details are given on the method we use to blend several action to a smooth facial animation. These techniques have been integrated in a C++ library which functionalities are described. Finally an interactive real-time application performing facial animation and speech using this library is described and graphical results are presented.