Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Conclusions
References

Representing Processes of Human Robot Collaboration

Profile image of Georg WeichhartGeorg Weichhart

2018

Abstract

S-BPM supports the representation of processes using two levels: the individual subject behaviour level and the subject interaction level. These two levels are helpful when it becomes necessary to specify the individual subject’s behaviour in di↵erent levels of details. We take a system-of-systems point of view and take a S-BPM specific look on interoperability. We use human-robot collaboration as a motivation and use-case to understand the interoperability requirements of adaptive system-of-systems. However, every type of human robot collaboration has di↵erent requirements not only for subject behaviour but also for the interaction level. We present the challenges and advantages of using SBPM in this setting.

Related papers

Context-dependent multimodal communication in human-robot collaboration
Balázs Pataki

Procedia CIRP

In the dynamic environment of human-robot collaboration, a key for boosting the efficiency of human workers is supporting them with contextdependent work instructions, delivered via communication modalities that suit the actual context. Workers, in turn, should be supported in controlling the robot or other components of the production system by using the most convenient modality, thus lifting the limitations of traditional interfaces as push buttons installed at fixed locations. We introduce a workflow for context-dependent multimodal communication in a collaborative work environment and present its implementation in a Human-Machine Interface Controller system.

View PDFchevron_right
Coordination mechanism for integrated design of Human-Robot Interaction scenarios
François Michaud

Paladyn, Journal of Behavioral Robotics, 2017

The ultimate long-term goal in Human-Robot Interaction (HRI) is to design robots that can act as a natural extension to humans. This requires the design of robot control architectures to provide structure for the integration of the necessary components into HRI. This paper describes how HBBA, a Hybrid Behavior-Based Architecture, can be used as a unifying framework for integrated design of HRI scenarios. More specifically, we focus here on HBBA’s generic coordination mechanism of behavior-producing modules, which allows to address a wide range or cognitive capabilities ranging from assisted teleoperation to selective attention and episodic memory. Using IRL-1, a humanoid robot equipped with compliant actuators for motion and manipulation, proximity sensors, cameras and a microphone array, three interaction scenarios are implemented: multi-modal teleoperation with physical guidance interaction, fetching-and delivering and tour-guiding.

View PDFchevron_right
Interactive Design for Behaviors of Robot Partners through Multi-modal Communication
Naoyuki Kubota

2010

View PDFchevron_right
Enabling End-Users in Designing and Executing of Complex, Collaborative Robotic Processes
Andreas Pichler

Applied System Innovation

Over the last years, capabilities of robotic systems have quantitatively and qualitatively improved. But going beyond isolated robotic systems, the integration and interoperability of robotic capabilities in complex work processes remains a major challenge. This lack of tools to integrate robots needs to be addressed on technical, semantic and organizational level. In the ROBxTASK research project, we developed an approach to support cooperation between different types of users in order to enable domain experts, with no robotic know-how, to work with robot-assisted workflows. By engineering robotic skills at a useful and usable level of abstraction for experts in different domains, we aim to increase re-usability of these skills on two different levels, (robotic) device level, and on level of application specific workflows. The researched prototype consists of a web platform, which allows (a) engineers to register (robotic) devices and the implemented skills of the devices, (b) doma...

View PDFchevron_right
Role Switching in Task-Oriented Multimodal Human-Robot Collaboration
Milos Zefran

2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 2020

In a collaborative task and the interaction that accompanies it, the participants often take on distinct roles, and dynamically switch the roles as the task requires. A domestic assistive robot thus needs to have similar capabilities. Using our previously proposed Multimodal Interaction Manager (MIM) framework, this paper investigates how role switching for a robot can be implemented. It identifies a set of primitive subtasks that encode common interaction patterns observed in our data corpus and that can be used to easily construct complex task models. It also describes an implementation on the NAO robot that, together with our original work, demonstrates that the robot can take on different roles. We provide a detailed analysis of the performance of the system and discuss the challenges that arise when switching roles in human-robot interactions.

View PDFchevron_right
Interaction Design Patterns for Adaptive Human-Agent-Robot Teamwork in High-Risk Domains
Mark Neerincx

Engineering Psychology and Cognitive Ergonomics, 2016

Integrating cognitive agents and robots into teams that operate in high-demand situations involves mutual and context-dependent behaviors of the human and agent/robot team-members. We propose a cognitive engineering method that includes the development of Interaction Design patterns for such systems as re-usable, theoretically and empirically founded, design solutions. This paper presents an overview of the background, the method and three example patterns.

View PDFchevron_right
From Tools to Teammates: Joint Activity in Human-Agent-Robot Teams
Jeffrey Bradshaw

2009

Coordination is an essential ingredient of joint activity in humanagent-robot teams. In this paper, we discuss some of the challenges and requirements for successful coordination, and briefly how we have used KAoS HART services framework to support coordination in a multi-team humanrobot field exercise.

View PDFchevron_right
Toward Human-Robot Collaboration
Kristian Simsarian

Recently robots have been launched as tour-guides in museums, lawnmowers, in-home vacuum cleaners, and as remotely operated machines in so-called distant, dangerous and dirty applications. While the methods to endow robots with a degree of autonomy have been a strong research focus, the methods for human-machine control have not been given as much attention. As autonomous robots become more ubiquitous, the methods we use to communicate task specification to them become more crucial. This thesis presents a methodology and a system for the supervisory collaborative control of a remote semi-autonomous mobile robot. The presentation centers on three main aspects of the work and offers a description of the system and the motivations behind the design. The supervisory system for human specification of robot tasks is based on a Collaborative Virtual Environment (CVE) which provides an effective framework for scalable robot autonomy, interaction and environment visualization. The system affords the specification of deictic commands to the semi-autonomous robot via the spatial CVE interface. Spatial commands can be specified in a manner that takes into account some specific everyday notions of collaborative task activity. Environment visualization of the remote environment is accomplished by combining the virtual model of the remote environment with video from the robot camera. Finally the system underwent a study with users that explored design and interaction issues within the context of performing a remote search task. Examples of study issues center on the presentation of the CVE, understanding robot competence, presence, control and interaction. One goal of the system presented in the thesis is to provide a direction in human-machine interaction from a form of direct control to an instance of human-machine collaboration.

View PDFchevron_right
System software architecture for advancing human robot interaction by cloud services and multi-robot cooperation
Anna Lekova

International Journal on Information Technologies and Security, 2024

Human-like interactions with robots based on Conversational AI facilitate assistance and teamwork in various contexts. Those interactions are further enhanced by utilizing physical presence and context from the robot's hardware. Robot cooperation is also especially useful, when software or hardware resources have to be shared in a multi-robot system. Therefore, we propose a modular software architecture for multi-robot cooperation that extends the integration of Conversational AI into Socially Assistive Robots, previously suggested by authors. It utilizes a flow-based approach that involves shared repositories and direct or message-driven communication to convey natural language transcriptions among robots in order to support their cooperation. By experiments we evaluated the cooperation between NAOqi based robots and Furhat robot. Our experimental results demonstrate architecture's modularity and adaptability to different cloud services, along with its effectiveness for interactions involving multiple robots.

View PDFchevron_right
A Hierarchical Architecture for Flexible Human-Robot Collaboration
Kourosh Darvish

2019

I thank all my friends who have been close to me and helped me to do this work. A special thanks is for azizam Chiara, who believed in me and supports me always. Thank you for all your invaluable inspiration and patience which made this work possible. To my beloved Mom and Dad, lontani chilometri ma neanche ad un passo dal cuore

View PDFchevron_right

References (39)

  1. Weichhart, G., Molina, A., Chen, D., Whitman, L., Vernadat, F.: Challenges and current developments for sensing, smart and sustainable enterprise systems. Computers in Industry 79 (2016) 34-46
  2. Weichhart, G., Stary, C., Vernadat, F.: Enterprise modelling for interoperable and knowledge-based enterprises. International Journal of Production Research (2018) accepted for publication.
  3. Gorod, A., White, B.E., Ireland, V., Gandhi, S.J., Sauser, B., eds.: Case Studies in System of Systems, Enterprise Systems, and Complex Systems Engineering. CRC Press, Taylor & Francis Group (2014)
  4. Boardman, J., Sauser, B.: System of systems -the meaning of of. In: System of Systems Engineering, 2006 IEEE/SMC International Conference on, IEEE (2006) 6 pp.-
  5. Romero, D., Bernus, P., Noran, O., Stahre, J., Fast-Berglund, Å.: The operator 4.0: Human cyber-physical systems & adaptive automation towards human-automation symbiosis work systems. In Nääs, I., Vendrametto, O., Mendes Reis, J., Gonçalves, R.F., Silva, M.T., von Cieminski, G., Kiritsis, D., eds.: Advances in Production Management Systems. Initiatives for a Sustainable World, Cham, Springer Inter- national Publishing (2016) 677-686
  6. Romero, D., Stahre, J., Wuest, T., Noran, O., Bernus, P., Fast-Berglund, Å., Gorecky, D.: Towards an operator 4.0 typology: a human-centric perspective on the fourth industrial revolution technologies. In: International Conference on Com- puters and Industrial Engineering (CIE46) Proceedings. (2016)
  7. Naudet, Y., Latour, T., Guédria, W., Chen, D.: Towards a systemic formalisation of interoperability. Computers in Industry 61 (2010) 176-185
  8. Ducq, Y., Chen, D., Doumeingts, G.: A contribution of system theory to sustain- able enterprise interoperability science base. Computers in Industry 63(8) (2012) 844 -857 Special Issue on Sustainable Interoperability: The Future of Internet Based Industrial Enterprises.
  9. Petrie, C.J., ed.: Enterprise Integration Modeling. The MIT Press, Cambridge, MA (1992)
  10. Panetto, H.: Towards a classification framework for interoperability of enterprise applications. International Journal of Computer Integrated Manufacturing 20(8) (December 2007) 727-740
  11. Dassisti, M., Jardim-Goncalves, R., Molina, A., Noran, O., Panetto, H., Zdravković, M.M.: Sustainability and interoperability: Two facets of the same gold medal. In Demey, Y.T., Panetto, H., eds.: On the Move to Meaningful In- ternet Systems: OTM 2013 Workshops. Volume 8186. Springer, Berlin Heidelberg (2013) 250-261
  12. Vernadat, F.B.: Interoperable enterprise systems: Principles, concepts, and meth- ods. Annual Reviews in Control 31(1) (2007) 137 -145
  13. Fox, M.S., Gruninger, M.J.: Enterprise modeling. AI Magazine 19(3) (1998) 109- 121
  14. Tu, Z., Zacharewicz, G., Chen, D.: A federated approach to develop enterprise interoperability. Journal of Intelligent Manufacturing 27(1) (2014) 11-31
  15. Vernadat, F.B.: Technical, semantic and organizational issues of enterprise inter- operability and networking. Annual Reviews in Control 34(1) (2010) 139 -144
  16. Stamper, R., Liu, K., Hafkamp, M., Ades, Y.: Understanding the roles of signs and norms in organizations -a semiotic approach to information systems design. Behaviour & Information Technology 19(1) (2000) 15-27
  17. Bénaben, F., Mu, W., Boissel-Dallier, N., Barthe-Delanoe, A.M., Zribi, S., Pingaud, H.: Supporting interoperability of collaborative networks through engineering of a service-based mediation information system (mise 2.0). Enterprise Information Systems 9(5-6) (2015) 556-582
  18. Lerchner, H., Stary, C.: An open s-bpm runtime environment based on abstract state machines. In Proper, H.A., Ralyté, J., Marchand-Maillet, S., Lin, K., eds.: 2014 IEEE 16th Conference on Business Informatics. (2014) 54-61
  19. Wohed, P., van der Aalst, W.M., Dumas, M., ter Hofstede, A.H., Russell, N.: On the suitability of bpmn for business process modelling. In Dustdar, S., Fiadeiro, J.L., Sheth, A., eds.: Proceedings 4th International Conference on Business Process Management. Volume 4102., Springer (2006) 161-176
  20. Fleischmann, A., Schmidt, W., Stary, C., Obermeier, S., Börger, E.: Subject- Oriented Business Process Management. Springer, Berlin Heidelberg (2012)
  21. Bastarz, F., Halek, P.: smart4sense2act: A smart concept for systemic performance management. In Schmidt, W., ed.: S-BPM ONE -Learning by Doing -Doing by Learning. Volume 213 of Communications in Computer and Information Science. Springer Berlin Heidelberg (2011) 109-114
  22. Meyer, N., Feiner, T., Radmayr, M., Blei, D., Fleischmann, A.: Dynamic catenation and execution of cross organisational business processes -the jcpex! approach. In Fleischmann, A., Schmidt, W., Singer, R., Seese, D., eds.: Subject-Oriented Business Process Management. Volume 138 of Communications in Computer and Information Science. Springer, Berlin Heidelberg (2011) 84-105
  23. Rothschädl, T.: Ad-hoc adaption of subject-oriented business processes at runtime to support organizational learning. In Stary, C., ed.: S-BPM ONE -Scientific Re- search. Volume 104 of Lecture Notes in Business Information Processing. Springer Berlin Heidelberg (2012) 22-32
  24. Neubauer, M., Krenn, F., Majoe, D., Stary, C.: Subject-orientation as design language for integration across organisational control layers. International Journal of Production Research 55 (2017) 3644-3656
  25. Weichhart, G., Wachholder, D.: On the interoperability contributions of s-bpm. In Nanopoulos, A., Schmidt, W., eds.: S-BPM ONE -Scientific Research. Volume 170 of LNBIP -Lecture Notes in Business Information Processing. Springer Inter- national Publishing (2014) 3-19
  26. Kannengiesser, U., Oppl, S.: Business processes to touch: Engaging domain experts in process modelling. In Daniel, F., Zugal, S., eds.: BPM Demo Session 2015. Volume 1418 of CEUR Workshop Proceedings., CEUR-WS.org (2015) 40-44
  27. Oppl, S.: Subject-oriented elicitation of distributed business process knowledge. In Schmidt, W., ed.: International Conference on Subject-Oriented Business Process Management. Volume 213 of Communications in Computer and Information Sci- ence., Springer-Verlag Berlin Heidelberg, Springer-Verlag Berlin Heidelberg (2011) 16-33
  28. Oppl, S., Stary, C.: Facilitating shared understanding of work situations using a tangible tabletop interface. Behaviour & Information Technology 33(6) (2014) 619-635 Published online: 02 Oct 2013.
  29. Oppl, S.: Articulation of subject-oriented business process models. In: Proceedings of the 7th International Conference on Subject-Oriented Business Process Manage- ment.
  30. S-BPM ONE '15, New York, NY, USA, ACM, ACM (2015) 2:1-2:11
  31. Meyer, N., Radmayr, M., Heininger, R., Rothschädl, T., Fleischmann, A.: Platform for managing and routing cross-organizational business processes on a network router. In Schmidt, W., ed.: S-BPM ONE -Learning by Doing -Doing by Learning. Volume 213 of Communications in Computer and Information Science (CCIS). Springer, Berlin Heidelberg (2011) 175-189
  32. Heininger, R.: Requirements for business process management systems supporting business process agility. In Oppl, S., Fleischmann, A., eds.: S-BPM ONE -Edu- cation and Industrial Developments. Volume 284 of Communications in Computer and Information Science. Springer, Berlin Heidelberg (2012) 168-180
  33. Weichhart, G., Feiner, T., Stary, C.: Implementing organisational interoperability -the sudden approach. Computers in Industry 61(2) (2010) 152 -160
  34. Neubauer, M., Stary, C., eds.: S-BPM in the Production Industry. Springer Inter- national Publishing, Switzerland (2017)
  35. Weichhart, G., Stary, C.: Interoperable process design in production systems. In Debruyne, C., Panetto, H., Weichhart, G., Bollen, P., Ciuciu, I., Vidal, M.E., Meersman, R., eds.: On the Move to Meaningful Internet Systems: OTM 2017
  36. Workshops. Number 10697 in Lecture Notes in Computer Science. Springer Inter- national, Cham, Switzerland (2018) 26-35
  37. Pichler, A., Akkaladevi, S.C., Ikeda, M., Hofmann, M., Plasch, M., Wögerer, C., Fritz, G.: Towards shared autonomy for robotic tasks in manufacturing. Procedia Manufacturing 11 (2017) 72-82
  38. Matthaiakis, S.A., Dimoulas, K., Athanasatos, A., Mparis, K., Dimitrakopoulos, G., Gkournelos, C., Papavasileiou, A., Fousekis, N., Papanastasiou, S., Michalos, G., Angione, G., Makris, S.: Flexible programming tool enabling synergy between human and robot. Procedia Manufacturing 11 (2017) 431 -440 27th International Conference on Flexible Automation and Intelligent Manufacturing, FAIM2017, 27- 30 June 2017, Modena, Italy.
  39. Müller, R., Vette, M., Mailahn, O.: Process-oriented task assignment for assembly processes with human-robot interaction. Procedia CIRP 44 (2016) 210 -215 6th CIRP Conference on Assembly Technologies and Systems (CATS).

Related papers

Workflow Representations for Human and Artificial Agent Collaborations
Georg Weichhart

In this initial research, we are taking a look at process representation and process modelling approaches for collaborative robotics. We take a system-of-systems view and apply work stemming from business process management, workflow systems to enable interoperable workflows. Initial requirements and research needs are identified for planning, executing interoperable process models, which are representing tasks shared among multiple human and artificial systems.

View PDFchevron_right
HRC-SoS: Human Robot Collaboration Experimentation Platform as System of Systems
Sarthak Arora

2019 14th Annual Conference System of Systems Engineering (SoSE), 2019

This paper presents an experimentation platform for human robot collaboration as a system of systems as well as proposes a conceptual framework describing the aspects of Human Robot Collaboration. These aspects are Awareness, Intelligence and Compliance of the system. Based on this framework case studies describing experiment setups performed using this platform are discussed. Each experiment highlights the use of the subsystems such as the digital twin, motion capture system, human-physiological monitoring system, data collection system and robot control and interface systems. A highlight of this paper showcases a subsystem with the ability to monitor human physiological feedback during a human robot collaboration task.

View PDFchevron_right
Supporting Complex Robot Behaviors with Simple Interaction Tools
David Gertman

Human Robot Interaction, 2007

Human-Robot Interaction 40 behavior, there is very little confusion. In actuality, the joystick has proved for many applications to be a valuable and effective interaction metaphor. The problem is that it is only appropriate for more direct levels of human-robot interaction. As the level of robot initiative and autonomy increases, the underlying metaphor for interaction must also change, resulting in a need for new and, at times, more sophisticated theories of robot behavior. The goal should not be to simplify the actual robot behaviors-complex environments and tasks require appropriately complex behaviors. Instead, the goal should be to simplify the user's mental model of robot behavior. This chapter advocates that alongside new autonomous capabilities, the user must also be given interaction tools that communicate a mental model of the robot and the task, enabling the user to predict and understand robot behavior and initiative. The necessary "theory of robot behavior" should not be communicated by providing users with a developer's perspective of the system. Behavioral sciences research indicates that passing on system complexity to the user will increase stress and failure rates while decreasing task efficiency (Gertman et al., 2005). Quite to the contrary, a great deal of craft may be required to filter and fuse the system data, abstracting away from the complexity to support a higher-level, functional understanding. The hard question is how exactly to accomplish this in a principled fashion. This chapter considers various components of interaction complexity common in humanrobot interfaces. With results from several previous HRI experiments, the paper examines means employed to reduce this complexity. These include a) perceptual abstraction and data fusion, b) the development of a common visualization and tasking substrate, and c) the introduction of interaction tools that simplify the theory of robot behavior necessary for the operator to understand, trust and exploit robot behavior. The elements of operator trust are considered in detail with emphasis on how intelligently facilitating information exchange and human and robot initiative can build appropriate trust. New approaches to facilitating human-robot initiative are discussed within the context of several real-world task domains including: robotic event photographer, ground-air teaming for mine detection, and an indoor search and detection task. Human-Robot Interaction 42 (short) human. The robot can rotate in place, and can also move forward and backward. The cameras can pan and tilt independently of the body.

View PDFchevron_right
The Task-State Coordination Pattern, with applications in Human-Robot-Interaction
Sven Wachsmuth

2011

We consider interaction a powerful enabling technology for robots in human environments. Besides taking commands or reporting, many other uses, such as interactive learning, are already being explored. However, HRI also poses systems engineering challenges that may hinder its adoption. To address these, we advocate a general coordination pattern for task execution: The Task-State Pattern. Crucially, it separates interaction coordination from task-level control, thus enabling independent, but integrated, development. In the pattern, tasks are represented using both a general, re-usable task coordination model and a task-type dependent specification. We have introduced a coordination model rich enough to support a powerful user experience, but still general enough to accomodate a variety of tasks, thus simplifying architecture and integration. Furthermore, because it is re-used in many places, it provides an attractive target for tool support.

View PDFchevron_right
Coordinating Shared Tasks in Human-Robot Collaboration by Commands
Roel Pieters

Frontiers in Robotics and AI, 2021

Human-robot collaboration is gaining more and more interest in industrial settings, as collaborative robots are considered safe and robot actions can be programmed easily by, for example, physical interaction. Despite this, robot programming mostly focuses on automated robot motions and interactive tasks or coordination between human and robot still requires additional developments. For example, the selection of which tasks or actions a robot should do next might not be known beforehand or might change at the last moment. Within a human-robot collaborative setting, the coordination of complex shared tasks, is therefore more suited to a human, where a robot would act upon requested commands.In this work we explore the utilization of commands to coordinate a shared task between a human and a robot, in a shared work space. Based on a known set of higher-level actions (e.g., pick-and-placement, hand-over, kitting) and the commands that trigger them, both a speech-based and graphical com...

View PDFchevron_right
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved