Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services, 2017
One key feature of TensorFlow includes the possibility to compile the trained model to run effici... more One key feature of TensorFlow includes the possibility to compile the trained model to run efficiently on mobile phones. This enables a wide range of opportunities for researchers and developers. In this tutorial, we teach attendees two basic steps to run neural networks on a mobile phone: Firstly, we will teach how to develop neural network architectures and train them in TensorFlow. Secondly, we show the process to run the trained models on a mobile phone.
Proceedings of the 2021 International Conference on Multimodal Interaction
ConAn-our graphical tool for multimodal conversation analysis-takes 360 degree videos recorded du... more ConAn-our graphical tool for multimodal conversation analysis-takes 360 degree videos recorded during multiperson group interactions as input. ConAn integrates state-of-the-art models for gaze estimation, active speaker detection, facial action unit detection, and body movement detection and can output quantitative reports both at individual and group level, as well as different visualizations that provide qualitative insights into group interaction.
Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
Previous work has shown that large high resolution displays (LHRDs) can enhance collaboration bet... more Previous work has shown that large high resolution displays (LHRDs) can enhance collaboration between users. As LHRDs allow free movement in front of the screen, an understanding of movement behavior is required to build successful interfaces for these devices. This paper presents Pac-Many; a multiplayer version of the classical computer game Pac-Man to study group dynamics when using LHRDs. We utilized smartphones as game controllers to enable free movement while playing the game. In a lab study, using a 4 m × 1 m LHRD, 24 participants (12 pairs) played Pac-Many in collaborative and competitive conditions. The results show that players in the collaborative condition divided screen space evenly. In contrast, competing players stood closer together to avoid benefits for the other player. We discuss how the nature of the task is important when designing and analyzing collaborative interfaces for LHRDs. Our work shows how to account for the spatial aspects of interaction with LHRDs to build immersive experiences.
Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
We present Pose-on-the-Go, a full-body pose estimation system that uses sensors already found in ... more We present Pose-on-the-Go, a full-body pose estimation system that uses sensors already found in today's smartphones. This stands in contrast to prior systems, which require worn or external sensors. We achieve this result via extensive sensor fusion, leveraging a phone's front and rear cameras, the user-facing depth camera, touchscreen, and IMU. Even still, we are missing data about a user's body (e.g., angle of the elbow joint), and so we use inverse kinematics to estimate and animate probable body poses. We provide a detailed evaluation of our system, benchmarking it against a professional-grade Vicon tracking system. We conclude with a series of demonstration applications that underscore the unique potential of our approach, which could be enabled on many modern smartphones with a simple software update. CCS CONCEPTS • Human-centered computing → Human computer interaction (HCI); Interaction techniques; Gestural input.
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
Figure 1. WorldGaze simultaneously opens the front and rear camera on smartphones. The front came... more Figure 1. WorldGaze simultaneously opens the front and rear camera on smartphones. The front camera is used to track the user's 3D head vector, which is then raycast into the world as seen by the rear camera. This allows users to intuitively define an object or region of interest using their head gaze, which voice assistants can utilize for more precise and natural interactions (right bottom).
Shortcut Gestures for Mobile Text Editing on Fully Touch Sensitive Smartphones
ACM Transactions on Computer-Human Interaction
While advances in mobile text entry enable smartphone users to type almost as fast as on hardware... more While advances in mobile text entry enable smartphone users to type almost as fast as on hardware keyboards, text-heavy activities are still not widely adopted. One reason is the lack of shortcut mechanisms. In this article, we determine shortcuts for text-heavy activities, elicit shortcut gestures, implement them for a fully touch-sensitive smartphone, and conduct an evaluation with potential users. We found that experts perform around 800 keyboard shortcuts per day, which are not available on smartphones. Interviews revealed the lack of shortcuts as a major limitation that prevents mobile text editing. Therefore, we elicited gestures for the 22 most important shortcuts for smartphones that are touch-sensitive on the whole device surface. We implemented the gestures for a fully touch-sensitive smartphone using deep learning and evaluated them in realistic scenarios to gather feedback. We show that the developed prototype is perceived as intuitive and faster than recent commercial a...
Uploads
Papers by Sven Mayer