People Watcher

Personal and Ubiquitous Computing

12th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA'06), 2006

We propose a novel tool, called objSampler, with which users can record and recall "encounters" with objects in ubiquitous computing environments. We encounter various things, individuals, and places in the real world either consciously, meaning encounters that we are aware of, or unconsciously, meaning those we are unaware of but physically close to them. While some of those encounters are particularly important or treasurable to us, the physical memory in our brain is often too volatile to remember them. In objSampler, we address this issue by providing a state-ofart hardware called objPipette that embeds a sensor node, an RF-ID reader, and a battery cell. Users can record conscious encounters with it by scanning RF-ID tags pasted on real world objects. In addition, the objPipette detects and records the places where the user is. Users can recall the recorded encounters by using a software support in objSampler, called objScope. This paper describes the design and implementation of objSampler. User study, which is also provided in this paper, shows that objSampler provides a unique and intuitive means to achieve the above goal.

IEEE Pervasive Computing, 2000

P ER VA SI V E computing 53

2006

We present the Context Watcher, a mobile application that enables mobile phone users to easily and unobtrusively share personal context data such as their location, heart rate, speed, or view, with their mutual consent. Not only can the data be shared, it can also be used as input for information services, to adapt applications to the context, or to automatically derive daily patterns and situational information, such as 'meeting with a supervisor', 'is a regular visitor' or 'having the best condition of all your friends'. The application is built atop of the MobiLife context management framework, a generic approach enabling context discovery, exchange and reasoning. In the context management framework, different entities, context providers, are exposed to and interact via the internet. The context management framework and its principles will be discussed in detail, as well as the different context providers that play a role in the Context Watcher application. The Context Watcher has a large user base of 100+ users, some of which are 24x7 power-users taking the Context Watcher wherever they go, enabling the research on daily patterns and situations to work with real-life data, resulting in powerful algorithms and better understanding of context and context-awareness.

We present HuddleLamp, a desk lamp with an integrated RGB-D camera that precisely tracks the movements and positions of mobile displays and hands on a table. This enables a new breed of spatially-aware multi-user and multi-device applications for around-the-table collaboration without an interactive tabletop. At any time, users can add or remove displays and reconfigure them in space in an adhoc manner without the need of installing any software or attaching markers. Additionally, hands are tracked to detect interactions above and between displays, enabling fluent cross-device interactions. We contribute a novel hybrid sensing approach that uses RGB and depth data to increase tracking quality and a technical evaluation of its capabilities and limitations. For enabling installation-free ad-hoc collaboration, we also introduce a web-based architecture and JavaScript API for future HuddleLamp applications. Finally, we demonstrate the resulting design space using five examples of cross-device interaction techniques.

2004

Location-enhanced applications use the location of people, places, and things to augment or streamline interaction. Location-enhanced applications are just starting to emerge in several different domains, and many people believe that this type of application will experience tremendous growth in the near future. However, it currently requires a high level of technical expertise to build location-enhanced applications, making it hard to iterate on designs. To address this problem we introduce Topiary, a tool for rapidly prototyping location-enhanced applications. Topiary lets designers create a map that models the location of people, places, and things; use this active map to demonstrate scenarios depicting location contexts; use these scenarios in creating storyboards that describe interaction sequences; and then run these storyboards on mobile devices, with a wizard updating the location of people and things on a separate device. We performed an informal evaluation with seven researchers and interface designers and found that they reacted positively to the concept.

ACM Transactions on Computer-Human Interaction, 2000

The proliferation of computing into the physical world promises more than the ubiquitous availability of computing infrastructure; it suggests new paradigms of interaction inspired by constant access to information and computational capabilities. For the past decade, application-driven research in ubiquitous computing (ubicomp) has pushed three interaction themes: natural interfaces, context-aware applications, and automated capture and access. To chart a course for future research in ubiquitous computing, we review the accomplishments of these efforts and point to remaining research challenges. Research in ubiquitous computing implicitly requires addressing some notion of scale, whether in the number and type of devices, the physical space of distributed computing, or the number of people using a system. We posit a new area of applications research, everyday computing, focussed on scaling interaction with respect to time. Just as pushing the availability of computing away from the traditional desktop fundamentally changes the relationship between humans and computers, providing continuous interaction moves computing from a localized tool to a constant companion. Designing for continuous interaction requires addressing interruption and resumption of interaction, representing passages of time and providing associative storage models. Inherent in all of these interaction themes are difficult issues in the social implications of ubiquitous computing and the challenges of evaluating ubiquitous computing research. Although cumulative experience points to lessons in privacy, security, visibility, and control, there are no simple guidelines for steering research efforts. Akin to any efforts involving new technologies, evaluation strategies form a spectrum from technology feasibility efforts to long-term use studies-but a user-centric perspective is always possible and necessary.

Moran, S., & Nakata, K. (2009). Ubiquitous Monitoring and Human Behaviour in Intelligent Pervasive Spaces. In Workshop on Social Mobile Web, as a part of the International Conference on Social Computing (pp. 1082–1087). Vancouver, Canada: IEEE CS.

Pythia is a prototype hybrid Mobile/Cloud service to ascertaining what the user of a mobile phone is currently doing. The service continuously captures and uploads context data to an analysis engine in the cloud. Early field testing showed that the service can categorize activities into working, at home, traveling, or shopping.

2015

The explosion of mobile applications and services presents challenges for evaluation and user study. One successful approach has been to deploy instrumented applications, logging their use over long periods of time. We present an expansion of this by remotely recording video and audio of use, while also capturing device and app context. In vivo combines five data collection techniques – screen recording, ambient audio recording, wearable cameras, data logging and distributed remote uploads. This data provides a range of insights and we discuss examples from previous work which reveal interaction design issues where interface confusions or task mismatches occur. We see how apps are integrated into ongoing activity and environment (such as how maps are used in situ), and how recorded conversations around and about apps may be used for evaluation purposes. We conclude by arguing that this combinative method helps us to move from considering app use in isolation, to studying app use in ...