Papers by Sebastian Ullrich
This paper contributes an integrated and flexible pipeline approach to sign language processing i... more This paper contributes an integrated and flexible pipeline approach to sign language processing in Virtual Reality (VR), that allows for interactive experimental evaluations with high ecological validity. Initial steps deal with real-time tracking and processing of manual gestures. Hand shape and motion data is rendered in virtual environments with varying spatial and representational configurations. Besides flexibility, the most important aspect is the seamless integration within a VR-based neuropsychological experiment software.
Abstract: Medical simulators are gaining importance because the experience and skills necessary t... more Abstract: Medical simulators are gaining importance because the experience and skills necessary to perform many of the medical procedures are difficult to obtain due to patient safety and ethical issues. With the development of graphic cards, stereographic and haptic devices, more VR-based simulators are being created. We are developing an interactive ultrasound image simulation that include deformations and needle visualization as part of a training simulator for the application of regional anesthesia.
In Anhang C befindet sich eine Auswahl der verwendeten Fragen. Zur einfacheren Auswertung sind di... more In Anhang C befindet sich eine Auswahl der verwendeten Fragen. Zur einfacheren Auswertung sind die Fragebögen web-basiert mit QPL realisiert worden. QPL (Questionnaire Programming Language) ist ein frei verfügbares Fragebogenerstellungssystem, wodurch zu einem gegeben Fragebogenquelltext automatisch PHP-Eingabeformulare und eine MySQL-Datenbank generiert werden [Gov04].
Abstract: Diese Arbeit befasst sich mit immersiver Modellierung. Hintergrund ist die Darstellung ... more Abstract: Diese Arbeit befasst sich mit immersiver Modellierung. Hintergrund ist die Darstellung von virtuellen Menschmodellen mit realistischer Hautdeformation durch eine mehrschichtige anatomische Muskulatursimulation. Dazu ist ein passendes Modellierungswerkzeug zur Erzeugung und Modifikation von Daten für die Simulation gesucht worden. Das aus dieser Anforderung entstandene MuscleAssemblyLab wird hier vorgestellt und durch ein vergleichendes Experiment evaluiert.
Palpation is a physical examination technique where objects, e.g., organs or body parts, are touc... more Palpation is a physical examination technique where objects, e.g., organs or body parts, are touched with fingers to determine their size, shape, consistency and location. Many medical procedures utilize palpation as a supplementary interaction technique and it can be therefore considered as an essential basic method. However, palpation is mostly neglected in medical training simulators, with the exception of very specialized simulators that solely focus on palpation, e.g., for manual cancer detection. In this article we propose a novel approach to enable haptic palpation interaction for virtual reality–based medical simulators. The main contribution is an extensive user study conducted with a large group of medical experts. To provide a plausible simulation framework for this user study, we contribute a novel and detailed interaction algorithm for palpation with tissue dragging, which utilizes a multi–object force algorithm to support multiple layers of anatomy and a pulse force algorithm for simulation of an arterial pulse. Furthermore, we propose a modification for an off–the–shelf haptic device by adding a lightweight palpation pad to support a more realistic finger grip configuration for palpation tasks. The user study itself has been conducted on a medical training simulator prototype with a specific procedure from regional anesthesia, which strongly depends on palpation. The prototype utilizes a co–rotational finite element approach for soft tissue simulation and provides bimanual interaction by combining the aforementioned techniques with needle insertion for the other hand. The results of the user study suggest reasonable face validity of the simulator prototype and in particular validate medical plausibility of the proposed palpation interaction algorithm.
In this paper we present a simulator for two-handed haptic interaction. As an application example... more In this paper we present a simulator for two-handed haptic interaction. As an application example, we chose a medical scenario that requires simultaneous interaction with a hand and a needle on a simulated patient. The system combines bimanual haptic interaction with a physics-based soft tissue simulation. To our knowledge the combination of finite element methods for the simulation of deformable objects with haptic rendering is seldom addressed, especially with two haptic devices in a non-trivial scenario. Challenges are to find a balance between real-time constraints and high computational demands for fidelity in simulation and to synchronize data between system components. The system has been successfully implemented and tested on two different hardware platforms: one mobile on a laptop and another stationary on a semi-immersive VR system. These two platforms have been chosen to demonstrate scaleability in terms of fidelity and costs. To compare performance and estimate latency, we measured timings of update loops and logged event-based timings of several components in the software.
emptyclip.de
In this paper, we present the results of a user study with a bimanual haptic setup. The goal of t... more In this paper, we present the results of a user study with a bimanual haptic setup. The goal of the experiment was to evaluate if Guiard's theory of the bimanual frame of reference can be applied to interaction tasks in virtual environments (VE) with haptic rendering. This theory proposes an influence of the non-dominant hand (NDH) on the dominant hand (DH). The experiment was conducted with multiple trials under two different conditions: bimanual and unimanual. The interaction task in this scenario was a sequence of pointing, alignment and docking sub-tasks for the dominant hand. In the bimanual condition, an asynchronous pointing task was added for the non-dominant hand. This additional task was primarily designed to bring the non-dominant hand closer to the other hand and thus enable the creation of a frame of reference. Our results show the potential of this task design extension (with NDH utilization). Task completion times are significantly lower in the bimanual condition compared to the unimanual case, without significant impact on overall precision. Furthermore, the bimanual condition shows better mean accuracy over several measures, e.g., lateral displacement and penetration depth. Additionally, subject performance was not only compared for all participants, but also between subgroups: medical vs. non-medical and gamer vs. non-gamer. User preference for a bimanual system over a unimanual system has been indicated with a post-test questionnaire.
IEEE Transactions on …, Jan 1, 2010
The aim of this paper is two-fold. First, it describes a scripting language for specifying commun... more The aim of this paper is two-fold. First, it describes a scripting language for specifying communicative behavior and interaction of computer-controlled agents ("bots") in the popular three-dimensional (3D) multi-user online world of "Second Life" and the emerging "OpenSimulator" project. While tools for designing avatars and in-world objects in Second Life exist, technology for non-programmer content creators of scenarios involving scripted agents is currently missing. Therefore, we have implemented new client software that controls bots based on the Multimodal Presentation Markup Language 3D (MPML3D), a highly expressive XML-based scripting language for controlling the verbal and non-verbal behavior of interacting animated agents. Second, the paper compares Second Life and OpenSimulator platforms and discusses the merits and limitations of each from the perspective of agent control. Here, we also conducted a small study that compares the network performance of both platforms.
Bildverarbeitung für die …, Jan 1, 2011
Segmentation of multiple muscles in magnetic resonance imaging (MRI) is challenging because of th... more Segmentation of multiple muscles in magnetic resonance imaging (MRI) is challenging because of the similar intensities of the tissue. In this paper, a novel approach is presented applying a scene-based discrete deformable model (simplex mesh). 3D segmentation is performed on a set of structures rather than on a single object. Relevant structures are modeled in a two-stage hierarchy from groups of clustered muscles (as they usually appear in MRI) to individual muscles. Collision detection is involved during mesh deformation to provide additional information of neighboring structures. The method is implemented in C++ within the Medical Imaging Interaction Toolkit (MITK) framework. As a proof of concept, we tested the approach on five datasets of the pelvis, three of which have been segmented manually. Indicating the potential impact of the method, we do not claim its general validity yet.
Studies in health …, Jan 1, 2011
In this paper we summarize the progress of the Web3D scene graph model, and associated standards,... more In this paper we summarize the progress of the Web3D scene graph model, and associated standards, specifically Extensible 3D (X3D) in the domain of medical simulation. Historically, the Web3D nodesets have focused on the representation and rendering of point, line or surface geometry. More recently, significant progress in X3D Volume rendering has been made available through the co-operative DICOM work item, n-Dimensional Presentation States. However, here we outline the need for a standard for simulation meshes and review several related approaches. As a result, we propose preliminary requirements for a simulation mesh standard and provide several use case scenarios of how Web3D and haptic technologies can aid the fulfillment of these requirements. We conclude with an X3D proposal to describe simulation meshes for soft (deformable) bodies.
Studies in health technology and …, Jan 1, 2011
In this paper, we investigated several approaches in literature that classify different aspects o... more In this paper, we investigated several approaches in literature that classify different aspects of medical simulators. We have merged these definitions to form a structured taxonomy. This new taxonomy should facilitate the design of new medical simulators and allow to analyze and classify existing simulators, algorithms, toolkits and hardware.
International journal of …, Jan 1, 2009
Purpose: The main purpose is to provide an intuitive VR-based training environment for regional a... more Purpose: The main purpose is to provide an intuitive VR-based training environment for regional anesthesia (RA). The research question is how to process subject-specific datasets, organize them in a meaningful way and how to perform the simulation for peripheral regions.
Methods: We propose a flexible virtual patient architecture and methods to process datasets. Image acquisition, image processing (especially segmentation), interactive nerve modeling and permutations (nerve instantiation) are described in detail. The simulation of electric impulse stimulation and according responses are essential for the training of peripheral RA and solved by an approach based on the electric distance.
Results: We have created an XML-based virtual patient database with several subjects. Prototypes of the simulation are implemented and run on multimodal VR hardware (e.g., stereoscopic display and haptic device). A first user pilot study has confirmed our approach.
Conclusion: The virtual patient architecture enables support for arbitrary scenarios on different subjects. This concept can also be used for other simulators. In future work, we plan to extend the simulation and conduct further evaluations in order to provide a tool for routine training for RA.
emptyclip.de
This paper reports about an ongoing research project, called
RASim. The goal of this project is t... more This paper reports about an ongoing research project, called
RASim. The goal of this project is to develop a virtual reality-based training tool for regional anaesthesia. A production pipeline to create patient-specific datasets has been established. A prototype of the simulator has been implemented on a virtual reality-platform and was evaluated in a first study. The current research and development is focused on soft-tissue simulation and haptics to improve the interaction. Furthermore, a bi-manual interaction scheme to combine palpation and virtual needle guidance is introduced as conceptual work and will be evaluated soon
Proceedings of 10th …, Jan 1, 2008
This paper describes a simulation framework for sensor-based systems based on Second Life, a popu... more This paper describes a simulation framework for sensor-based systems based on Second Life, a popular virtual 3D multi-user online world. With this platform, the components of sensor-based systems can be mapped to, or represented by, virtual devices and objects. The intuitive user interface of Second Life (SL) and its comprehensive visualization capability support evaluation tasks of ubiquitous computing applications. Developers, as avatars, can directly control and manipulate virtual devices. In this way, different settings of sensor-based systems can be tested, and interactively improved. A bidirectional interface between sensor-based systems and Second Life has been implemented to demonstrate Second Life as a testbed for an RFID-based positioning system.
Applications in simulation or visualization often rely on data from multiple sources (e.g., MRI s... more Applications in simulation or visualization often rely on data from multiple sources (e.g., MRI scans, extracted anatomical objects in a 3D representation). However, structured data organization is either non-existent (e.g., simply stored in folders), proprietary or specific (e.g., DICOM archives). In this paper, a novel extensible approach to organize and access arbitrary medical data is proposed, called Medical Data Organization with XML (MEDOX). The concept is based on a lightweight XML database, complementary to recent MedX3D standard developments. Data files are stored in a file system and referenced along with additional meta data in an XML file. Basic versioning of data objects is also supported. A simple C/C++ library was implemented to abstract the access patterns to the database and to avoid the error-prone manual manipulation of the XML file. As a proof of concept, it has been applied to regional anesthesia simulation in virtual environments.
British journal of …, Jan 1, 2009
Background: The safe performance of regional anaesthesia (RA) requires theoretical knowledge and ... more Background: The safe performance of regional anaesthesia (RA) requires theoretical knowledge and good manual skills. Virtual reality (VR)-based simulators may offer trainees a safe environment to learn and practice different techniques. However, currently available VR simulators do not consider individual anatomy, which limits their use for realistic training. We have developed a VR-based simulator that can be used for individual anatomy and for different anatomical regions.
Methods: Individual data were obtained from magnetic resonance imaging (MRI) and magnetic resonance angiography (MRA) without contrast agent to represent morphology and the vascular system, respectively. For data handling, registration, and segmentation, an application based on the Medical Imaging Interaction Toolkit was developed. Suitable segmentation algorithms such as the fuzzy c-means clustering approach were integrated, and a hierarchical tree data structure was created to model the flexible anatomical structures of peripheral nerve cords. The simulator was implemented in the VR toolkit ViSTA using modules for collision detection, virtual humanoids, interaction, and visualization. A novel algorithm for electric impulse transmission is the core of the simulation.
Results: In a feasibility study, MRI morphology and MRA were acquired from five subjects for the inguinal region. From these sources, three-dimensional anatomical data sets were created and nerves modelled. The resolution obtained from both MRI and MRA was sufficient for realistic simulations. Our high-fidelity simulator application allows trainees to perform virtual peripheral nerve blocks based on these data sets and models.
Conclusions: Subject-specific training of RA is supported in a virtual environment. We have adapted segmentation algorithms and developed a VR-based simulator for the inguinal region for use in training for different peripheral nerve blocks. In contrast to available VR-based simulators, our simulation offers anatomical variety.
Bildverarbeitung für die …, Jan 1, 2007
In this paper, a novel virtual reality-based simulator for regional anaesthesia is presented. Ind... more In this paper, a novel virtual reality-based simulator for regional anaesthesia is presented. Individual datasets of patients with nerve cords are created from medical scans with the help of advanced segmentation and registration algorithms. Techniques for interaction and immersive visualization are utilized by the simulator to improve training of medical residents.
Uploads
Papers by Sebastian Ullrich
Methods: We propose a flexible virtual patient architecture and methods to process datasets. Image acquisition, image processing (especially segmentation), interactive nerve modeling and permutations (nerve instantiation) are described in detail. The simulation of electric impulse stimulation and according responses are essential for the training of peripheral RA and solved by an approach based on the electric distance.
Results: We have created an XML-based virtual patient database with several subjects. Prototypes of the simulation are implemented and run on multimodal VR hardware (e.g., stereoscopic display and haptic device). A first user pilot study has confirmed our approach.
Conclusion: The virtual patient architecture enables support for arbitrary scenarios on different subjects. This concept can also be used for other simulators. In future work, we plan to extend the simulation and conduct further evaluations in order to provide a tool for routine training for RA.
RASim. The goal of this project is to develop a virtual reality-based training tool for regional anaesthesia. A production pipeline to create patient-specific datasets has been established. A prototype of the simulator has been implemented on a virtual reality-platform and was evaluated in a first study. The current research and development is focused on soft-tissue simulation and haptics to improve the interaction. Furthermore, a bi-manual interaction scheme to combine palpation and virtual needle guidance is introduced as conceptual work and will be evaluated soon
Methods: Individual data were obtained from magnetic resonance imaging (MRI) and magnetic resonance angiography (MRA) without contrast agent to represent morphology and the vascular system, respectively. For data handling, registration, and segmentation, an application based on the Medical Imaging Interaction Toolkit was developed. Suitable segmentation algorithms such as the fuzzy c-means clustering approach were integrated, and a hierarchical tree data structure was created to model the flexible anatomical structures of peripheral nerve cords. The simulator was implemented in the VR toolkit ViSTA using modules for collision detection, virtual humanoids, interaction, and visualization. A novel algorithm for electric impulse transmission is the core of the simulation.
Results: In a feasibility study, MRI morphology and MRA were acquired from five subjects for the inguinal region. From these sources, three-dimensional anatomical data sets were created and nerves modelled. The resolution obtained from both MRI and MRA was sufficient for realistic simulations. Our high-fidelity simulator application allows trainees to perform virtual peripheral nerve blocks based on these data sets and models.
Conclusions: Subject-specific training of RA is supported in a virtual environment. We have adapted segmentation algorithms and developed a VR-based simulator for the inguinal region for use in training for different peripheral nerve blocks. In contrast to available VR-based simulators, our simulation offers anatomical variety.