Design of a Knowledge-Based Agent as a Social Companion
2017, Procedia Computer Science
https://doi.org/10.1016/J.PROCS.2017.11.119Abstract
We present work in progress on an intelligent embodied conversation agent that is supposed to act as a social companion with linguistic and emotional competence in the context of basic and health care. The core of the agent is an ontology-based knowledge model that supports flexible reasoning-driven conversation planning strategies. A dedicated search engine ensures the provision of background information from the web, necessary for conducting a conversation on a specific topic. Multimodal communication analysis and generation modules analyze respectively generate facial expressions, gestures and multilingual speech. The assessment of the prototypical implementation of the agent shows that users accept it as a natural and trustworthy conversation counterpart. For the final release, all involved technologies will be further improved and matured.
References (32)
- Anderson, K., André, E., Baur, T., Bernardini, S., Chollet, M., Chryssadou, E., Damian, I., Ennis, C., Egges, A., Gebhard, P., Jones, H., Ochs, M., Pelachaud, C., Porayska-Pomsta, K., Rizzo, P., Sabouret, N.: The TARDIS framework: Intelligent virtual agents for social coaching in job interviews. In: Reidsma, D., Katayose, H., Nijholt, A. (eds.) ACE, vol. LNCS, 8253. 2013; p. 476-491. Springer, Heidelberg.
- Ballesteros, M., Bohnet, B., Mille, S., Wanner, L.: Data-driven deep-syntactic dependency parsing. Natural Language Engineering. 2016; 22(6):939-974.
- Ballesteros, M., Bohnet, B., Mille, S., Wanner, L.: Data-driven sentence generation with non-isomorphic trees. In: Proceedings of the Conference of the NAACL: Human Language Technologies; 2015. p. 387-397.
- Baldassare, M., Rosenfield, S., and Rook, K. The types of social relations predicting elderly well-being. Res on Aging. 1984. 6(4):549 -559.
- Baur, T., Mehlmann, G., Damian, I., Gebhard, P., Lingenfelser, F., Wagner, J., Lugrin, B., André E.: Context-Aware Automated Analysis and Annotation of Social Human-Agent Interactions. ACM Transactions on Interactive Intelligent Systems. 2015; 5(2).
- Bohnet, B., Wanner, L. Open source graph transducer interpreter and grammar development environment. In: Proceedings of the International Conference on Language Resources and Evaluation; 2010.
- Domínguez, M., Farrús, M., Burga, A., Wanner, L.: Using hierarchical information structure for prosody prediction in content-to-speech application. In: Proceedings of the 8thInternational Conference on Speech Prosody; 2016.
- Ekman, P., Rosenberg, E.L. What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS). Oxford University Press, USA; 1997.
- Gangemi, A.: The Semantic Web. In: Proceedings of the 4th International Semantic Web Conference; 2005, p. 262 -27
- Gebhard, P., Mehlmann, G.U., Kipp, M.: Visual SceneMaker: A Tool for Authoring Interactive Virtual Characters. Journal of Multimodal User Interfaces: Interacting with Embodied Conversational Agents, Springer-Verlag. 2012; 6(1-2):3-11.
- Gunes, H., Schuller, B.: Categorical and dimensional affect analysis in continuous input: Current trends and future directions. Image and Vision Computing 2013; 31(2):120-136.
- Hofstede, G.H., Hofstede, G. Culture's consequences: Comparing values, behaviors, institutions and organizations across nations. Sage. 2001.
- Hyde, J., Carter, E.J., Kiesler, S., Hodgins, J.K.: Assessing naturalness and emotional intensity: a perceptual study of animated facial motion. In: Proceedings of the ACM Symposium on Applied Perception. 2014; p 15-22. ACM.
- Hyde, J., Carter, E.J., Kiesler, S., Hodgins, J.K.: Using an interactive avatar's facial expressiveness to increase persuasiveness and socialness. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 2015; p. 1719 -1728. ACM.
- Lingenfelser, F.,Wagner, J., André, E., McKeown, G., Curran, W.: An event driven fusion approach for enjoyment recognition in real-time. In: Proceedings of the Multimedia Conference. 2014; p. 377-386.
- Mehlmann, G., Janowski, K., André, E.: Modeling Grounding for Interactive Social Companions. Journal of Artificial Intelligence: Social Companion Technologies. 2016. 30(1):45-52.
- Mehlmann, G., Janowski, K., Baur, T., Häring, M., André, E., Gebhard, P. Exploring a Model of Gaze for Grounding in HRI. In: Proceedings of the 16th International Conference on Multimodal Interaction. 2014; p. 247-254. ACM.
- Ochs, M., Pelachaud, C.: Socially Aware Virtual Characters: The Social Signal of Smiles. IEEE Signal Processing Magazine. 2013; 30(2):128- 132.
- Pfeifer Vardoulakis, L., Ring, L., Barry, B., Sidner, C., Bickmore, T.: Designing relational agents as long term social companions for older adults. In: Proceedings of the 12th International Conference on Intelligent Virtual Agents. 2012.
- Pickett Y, Raue, PJ, Bruce, ML. Late-life depression in home healthcare. J Aging Health 2012; 8(3): 273-284.
- Posner, J., Russell, J., Peterson, B.: The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development and psychopathology. Development and psychopathology. 2005; 17(3).
- Riaño, D., Real, F., Campana, F., Ercolani, S., Annicchiarico, R.: An ontology for the care of the elder at home. In: Proceedings of the 12th Conference on Artifcial Intelligence in Medicine: Artifcial Intelligence in Medicine. 2009; p. 235-239. AIME '09, Springer-Verlag, Berlin.
- Savran, A., Sankur, B., Bilge, M.T.: Regression-based intensity estimation of facial action units. Image and Vision Computing 2012; 30(10):774 -784.
- Shaw, R., Troncy, R., Hardman, L.: Lode: Linking open descriptions of events. In: Proceedings of the 4th Asian Conference on the Semantic Web. 2009; p. 153-167. Shanghai, China.
- Sorkin, D., Rook, K.S. and Lu, J.L.: Loneliness, lack of emotional support, lack of companionship, and the likelihood of having a heart condition in an elderly sample. Ann. Behav. Med. 2002. 24: 290-298.
- Leo Wanner et al. / Procedia Computer Science 121 (2017) 920-926
- Leo Wanner et al./ Procedia Computer Science 00 (2017) 000-000 7
- Vlachantoni, A., Shaw, R., Willis, R., Evandrou, M., Falkingham, J., Luf, R.. Measuring unmet need for social care amongst older people. Population Trends. 2011; 145:1-17.
- Wagner, J., Lingenfelser, F., André, E.: Building a robust system for multimodal emotion recognition. Emotion Recognition: A Pattern Analysis Approach. 2015; p. 379-419. John Wiley & Sons, Hoboken, NJ.
- Wanner, L., Bohnet, B., Bouayad-Agha, N., Lareau, F., Nicklass, D.: MARQUIS: Generation of user-tailored multilingual air quality bulletins. Applied Artifcial Intelligence. 2010; 24(10):914-952.
- Yasavur, U., Lisetti, C., Rishe, N.: Lets talk! Speaking virtual counselor offers you a brief intervention. Journal of Multimodal User Interfaces. 2014; 8(4):381-398.
- Zeng, Z., Pantic, M., Roisman, G., Huang, T.: A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE transactions on pattern analysis and machine intelligence. 2009; 31(1):39-58.