Design of a Pressure Sensitive Floor for Multimodal Sensing

Proceedings of the 2005 ACM SIGCHI International Conference on Advances in computer entertainment technology, 2005

This paper explores the design of a reconfigurable large-area high-resolution pressure sensing floor to help study human dance movement. By measuring the pressure of a user interacting with the system, our device is able to provide real-time knowledge about both the location of the performer on the floor as well as the amount and distribution of force being exerted on the floor. This system has been designed to closely integrate and synchronize with external systems including marker-based motion capture systems, audio-sensing equipment and video-sensing technology, thus allowing for robust multimodal sensing of a subject in the integrated environment. Furthermore, the mats comprising the floor can be readily re-arranged in order to allow for a large number of configurations. Some other possible applications of the pressure sensing floor include virtual reality based entertainment systems or video game control interfaces as well as rehabilitation projects for disabled people with foot or motor-control disorders.

Lecture Notes in Computer Science

This paper addresses the design of a large area, high resolution, networked pressure sensing floor with primary application in movement-based human-computer interaction (M-HCI). To meet the sensing needs of an M-HCI system, several design challenges need to be overcome. Firstly, high frame rate and low latency are required to ensure real-time human computer interaction, even in the presence of large sensing area (for unconstrained movement in the capture space) and high resolution (to support detailed analysis of pressure patterns). The optimization of floor system frame rate and latency is a challenge. Secondly, in many cases of M-HCI there are only a small number of subjects on the floor and a large portion of the floor is not active. Proper data compression for efficient data transmission is also a challenge. Thirdly, locations of disjoint active floor regions are useful features in many M-HCI applications. Reliable clustering and tracking of active disjoint floor regions poses as a challenge. Finally, to allow M-HCI using multiple communication channels, such as gesture, pose and pressure distributions, the pressure sensing floor needs to be integrable with other sensing modalities to create a smart multimodal environment. Fast and accurate alignment of floor sensing data in space and time with other sensing modalities is another challenge. In our research, we fully addressed the above challenges. The pressure sensing floor we developed has a sensing area of about 180 square feet, with a sensor resolution of 6.25 sensels/in 2 . The system frame rate is up to 43 Hz with average latency of 25 ms. A simple but efficient data compression scheme is in place. We have also developed a robust clustering and tracking procedure for disjoint active floor regions using the mean-shift algorithm. The pressure sensing floor can be seamlessly integrated with a marker based motion capture system with accurate temporal and spatial alignment. Furthermore, the modular and scalable structure of the sensor floor allows for easy installation to real rooms of irregular shape. The pressure sensing floor system described in this paper forms an important stepping stone towards the creation of a smart environment with context aware data processing algorithms which finds extensive applications beyond M-HCI, e.g. diagnosing gait pathologies and evaluation of treatment.

Human Computer Interaction, 2008

Human-computer interaction is a discipline concerned with the design, evaluation and implementation of interactive computing systems for human use. Humans communicate with each other, intentionally or unintentionally, using various interpersonal communication modes such as static and dynamic full-body, limb, and hand gestures, facial expressions, speech and sounds, and haptics, just to name a few. It is natural to design human-computer interaction systems with which users can communicate using these interpersonal communication modes. To this end, multimodal human-computer interaction (MMHCI) systems are receiving increasing attention recently. An overview of the recent advances of MMHCI can be found in . Our research mainly focuses on movement analysis based on visual and pressure sensing for movement based MMHCI, which read the movement of user(s), and respond accordingly through real-time visual and audio feedback. Such movement based MMHCI systems have immediate applications in a number of areas with significant impact on our daily lives, including biomedical, e.g. rehabilitation of stroke patients , culture and arts, e.g. studying patterns and cues in complex dance performances, and interactive dance performances , K-12 education, e.g. collaborative and embodied learning , sports (e.g. analyzing and improving athletic performance based on weight distributions), and security (e.g. movement based smart surveillance systems), just to name a few. Movement based MMHCI mainly deals with looking at dynamic characteristics of a person or a group of people such as joint angles, position of body parts, force and torque associated with limb movements, instantaneous velocity, acceleration and direction of body motion. In order to enable such a system to understand the user's movement robustly and accurately, it is important to augment the user's environment with novel sensors for accurate detection and estimation of the above movement qualities. It is worthy to understand that all the above movement qualities have underlying shape and/or effort attached which forms vital degrees of freedom for sensing modalities. Optical motion capture systems have become the obvious choice of researchers and technologists today for visual sensing of movement. However visual sensing alone is not sufficient for holistic www.intechopen.com Human-Computer Interaction 136 inference of human movement since it can comprehend only shapes e.g. joint angles, orientation of body parts associated with human movement and give no clue about effort. Also visual sensing suffers from occlusion. Haptic sensing such as pressure sensing becomes inevitable for the above reasons as it aids to understand and comprehend the motivation driven physical effort attached to every movement and thereby exploring the inherent nature of the human body as a powerful communication medium. Taking all the above factors into account, multimodal movement based human computer interaction system has been envisioned using both the pressure sensing floor (haptic) and motion capture system (visual) in order to perform holistic human movement analysis. The motion capture system that we use is commercially available and has been purchased for our research. However the pressure sensing floor is an in-house system developed specifically to address the research problem which thereby forms the core focus of this chapter. In this chapter, we present the system level description of pressure sensing floor followed by a discussion on hardware and software developments. Then we discuss the design methodologies for integration of the floor system with the marker based motion capture system as a first step towards the creation of an integrated multimodal environment. Pressure sensing system design targeting human computer interaction applications should confirm to certain requirements. In order to meet the sensing needs of such an application several design challenges need to be overcome. Firstly, the pressure sensing system should have a large sensing area to allow for unconstrained movement in the capture space. Secondly, high sensor densities are required for precise pressure localization and detailed analysis of pressure patterns. Thirdly high frame rate and low sensing latency are indeed critical for real time human computer interaction to capture rapidly changing human activities. It is worth mentioning here that there is a performance trade off between frame rate/ sensing latency and sensing area/sensor densities. Large sensing area with high sensor densities results in large number of sensors for scanning and data acquisition thereby decreasing the maximum achievable frame rate and increasing the sensing latency. Hence the performance optimization of the pressure sensing system to ensure large sensing area, high sensor resolution at reasonably good frame rate and low sensing latency is a major challenge. Fourthly, in many cases, there are only few users and large portion of the sensing space is not active at all. So proper data compression scheme to avoid network congestion and effectively utilize the given bandwidth poses a challenge. Fifthly, smart sensing systems should be inevitably equipped with context aware capabilities to sense the state of the environment and users and make a perception regarding the context of the environment or the user. Reliable person location tracking by clustering and tracking of active disjoint floor regions forms a vital part of perceiving context and emerges as a major implementation challenge. Finally, to allow movement based human computer interaction using multiple communication channels, such as gesture, pose and pressure distributions, the pressure sensing floor needs to be integrable with other sensing modalities to create a smart multimodal environment. Fast and accurate alignment of floor sensing data in space and time with other sensing modalities is another challenge. Furthermore, a need exists for a www.intechopen.com Design Optimization of Pressure Sensing Floor for Multimodal Human-Computer Interaction 137 design of a modular and scalable system to allow for easy expansion and reconfiguration to suit external environments of different shapes and sizes. In related prior work, various pressure sensing systems had been developed to capture and view pressure information associated with human movement across a floor. A detailed performance comparison study of those existing pressure sensing systems in terms of the above mentioned desired features are listed in Table .

ACM Transactions on Interactive Intelligent Systems, 2015

Smart environments are now designed as natural interfaces to capture and understand human behavior without a need for explicit human-computer interaction. In this article, we present a general-purpose architecture that acquires and understands human behaviors through a sensing floor. The pressure field generated by moving people is captured and analyzed. Specific actions and events are then detected by a low-level processing engine and sent to high-level interfaces providing different functions. The proposed architecture and sensors are modular, general-purpose, cheap, and suitable for both small- and large-area coverage. Some sample entertainment and virtual reality applications that we developed to test the platform are presented.

2013

We explore how to track people and furniture based on a high-resolution pressure-sensitive floor. Gravity pushes people and objects against the floor, causing them to leave imprints of pressure distributions across the surface. While the sensor is limited to sensing direct contact with the surface, we can sometimes conclude what takes place above the surface, such as users’ poses or collisions with virtual objects. We demonstrate how to extend the range of this approach by sensing through passive furniture that propagates pressure to the floor. To explore our approach, we have created an 8 m2 back-projected floor prototype, termed GravitySpace, a set of passive touch-sensitive furniture, as well as algorithms for identifying users, furniture, and poses. Pressure-based sensing on the floor offers four potential benefits over camera- based solutions: (1) it provides consistent coverage of rooms wall-to-wall, (2) is less susceptible to occlusion between users, (3) allows for the use of simpler recognition algorithms, and (4) intrudes less on users’ privacy.