Working on the Threshold of the Human Error in Maintenance Tasks

Applied Sciences

Maintenance has been one of the most important domains for augmented reality (AR) since its inception. AR applications enable technicians to receive visual and audio computer-generated aids while performing different activities, such as assembling, repairing, or maintenance procedures. These procedures are usually organized as a sequence of steps, each one involving an elementary action to be performed by the user. However, since it is not possible to automatically validate the users actions, they might incorrectly execute or miss some steps. Thus, a relevant open problem is to provide users with some sort of automated verification tool. This paper presents a system, used to support maintenance procedures through AR, which tries to address the validation problem. The novel technology consists of a computer vision algorithm able to evaluate, at each step of a maintenance procedure, if the user correctly completed the assigned task or not. The validation occurs by comparing an image o...

IEEE Computer Graphics and Applications, 2011

Recent statistics on causes of aviation accidents and incidents demonstrate that to increase air-transportation safety, we must reduce human errors' impact on operations. So, the industry should first address human factors related to people in stressful roles to significantly minimize such errors. In particular, aviation maintenance employees work under high-pressure conditions- that is, they're under strict time constraints and must adhere to stringent guidelines. Because of such constraints, they might be prone to making errors. Unfortunately, many of these errors might not become apparent until an accident occurs. Although maintenance errors are a recognized threat to aviation safety, there are few simulation and computer-based tools for managing human factor issues in this field. The main advantages in using computer-based systems to train or support technicians are that computers don't forget and that they can help humans clearly understand facts. Such features can help reduce errors due to procedure violations, misinterpretation of facts, or insufficient training. Toward that end, augmented reality (AR) is a promising technology to build advanced interfaces using interactive and wearable visualization systems to implement new methods to display documentation as digital data and graphical databases. Nevertheless, many factors-such as cumbersome hardware, the need to put markers on the aircraft, and the need to quickly create digital content-seem to hinder its effective implementation in industry.

Robotics and Computer-Integrated Manufacturing, 2019

This paper describes an Augmented Reality Maintenance System (ARMS) that can provide adaptive support and feedback to a user during maintenance. An innovative wrist-based haptic tracking methodology is implemented to provide maintenance step awareness to the maintenance operator when he/she is working within the maintenance workspace. This is integrated with an image-based step detection module to facilitate the determination on whether a step has been conducted correctly. To further adapt to the user's visualisation requirements during maintenance, a novel adaptive human-machine interface module has been incorporated. The result is a seamless AR-assisted maintenance platform that can better cater to the needs of the operators during maintenance by providing useful feedback to the operators. The system has been demonstrated through a case study to highlight the viability of the methodology in actual industrial scenarios.

Human Computer Interaction: New Developments, 2008

2006

One of the main weaknesses with traditional computing is the fact that the numerical world of the computer is decoupled from the user's real world. The application of Augmented Reality (AR) can provide interactive systems in which real objects and computer data are combined in a cohesive way. This new paradigm has many potential applications in various fields, in particular in the maintenance domain. It allows the user to see computer generated virtual objects superimposed to the real world through the see-through Head Mounted Display (HMD). The technician of maintenance, when using this system, can interact with the virtual world and have additional information, such as instruction for performing maintenance tasks in form of text messages, images, 3-D models of pieces or audio such as speech instruction. In this paper, we propose a design process of the maintenance system focused on the analysis of the interaction between the user, the system and the real world. This area is based on the UML notation. The use of UML represents our ergonomic and software design process basis for AR systems. This process also is based on ergonomic characteristics study within a UML system description and on the hybrid PAC-Amodeus model architecture adaptation for the AR systems.

The 17th International Conference on Virtual-Reality Continuum and its Applications in Industry, 2019

The machinery used in industrial applications, such as in agriculture, construction, and forestry, are increasingly equipped with digital tools that aim to aid the operator in task completion, improved productivity, and enhanced safety. In addition, as machines are increasingly connected, there are even more opportunities to integrate external information sources. This situation provides a challenge in mediating the information to the operator. One approach that could be used to address this challenge is the use of augmented reality. This enables the system-generated information to be combined with the user's perception of the environment. It has the potential to enhance the operators' awareness of the machine, the surroundings, and the operation that needs to be performed. In this paper, we review the current literature to present the state of the art, discuss the possible bene ts, and the use of augmented reality in heavy machinery. CCS CONCEPTS • Human-centered computing → Mixed / augmented reality.

2012

This research proposes a system to aid maintenance, inspection and training procedures involving electrical equipment. The aid happens with the addition of visual highlights, audio, video and image files, and through the removal of unwanted parts of the target equipment that can divert the operator’s focus from the desired place in a specific moment. This system will run on a wearable computer that will show the information to the operator through a display attached to his/her head. This thesis covers the proposal of a tool to build new procedures too, which uses just pictures of the target equipment, and a wearable platform to be used in the field.

Recent Trends of Mobile Collaborative Augmented Reality Systems , 2011

Training for service technicians in maintenance tasks is a classical ­application field of Augmented Reality explored by different research groups. Mostly technical aspects (tracking, visualization etc.) have been in focus of this research field. In this chapter we present results of interdisciplinary research based on the fusion of cognitive science, psychology and computer science. We focus on analyzing the improvement of Augmented Reality based training of skills which are relevant for maintenance and assembly tasks. The skills considered in this work comprise sensorimotor skills as well as cognitive skills. Different experiments have been conducted in order to find recommendations for the design of Augmented Reality training systems which overcome problems of existing approaches. The suggestions concern the fields of content visualization and multimodal feedback.

2018 3rd International Conference on Pattern Analysis and Intelligent Systems (PAIS), 2018

In this article, we present ARPM (Augmented Reality-based Preventive Maintenance) system, which aims at offering to the technician of a Cement Factory a virtual assistance in real time using various forms of 3D models, images or even texts, in order to help him to maintain and inspect his equipment. This system not only offers the possibility of reducing costs of the preventive maintenance, but also allows monitoring and even anticipating failures. In order to ensure a reliable maintenance operation, we adapted our system to be compatible with the CMMS system (Computerized Maintenance Management System) already exists in the Cement factory.

International Journal of Industrial Ergonomics, 2005

This exploratory study was carried out in the early phases of an R & D project for the implementation of an augmented reality-based (AR-based) job aid for automotive service technicians (ASTs). Prior to the study, key stakeholder groups had little clarity on the actual goal of the future system. Strong emphasis was placed on providing technicians with step-by-step guidance during repairs to new models of vehicles. AR was also assumed to be useful as a visualization aid for inaccessible vehicle parts. An ergonomic study was proposed to help clarify the high-level end users' requirements. The study, on 11 mechanics, was done using the critical incident technique. The incidents were collected through interviews. The results showed that the major difficulty in ASTs' activity was diagnosing electronic components on new models of vehicles. Other critical factors appeared in technicians' verbal reports, e.g., physical and organisational difficulties. The visualization of inaccessible vehicle parts, which was stakeholders' primary concern, only seemed problematic in one case. On the basis of these results, we discuss the applicability of AR to ASTs' work as well as a few perspectives of the study. www.elsevier.com/locate/ergon 0169-8141/$ -see front matter r (J.-M. Burkhardt), christine.megard@cea.fr (C. Me´gard), pierre.ehanno@renault.com (P. Ehanno).