Chunyuan Liao

Linking digital media to physical documents: Comparing content-based and marker-based tags

Paper is static but it is also light, flexible, robust, and has high resolution for reading docum... more Paper is static but it is also light, flexible, robust, and has high resolution for reading documents in various scenarios. Digital devices will likely never match the flexibility of paper, but come with all of the benefits of computation and networking. Tags provide a simple means of bridging the gap between the two media to get the most out of

Download

TrafficView: Towards a Scalable Traffic Monitoring System

Vehicles are part of people's life in modern society, into which more and more high-tech devi... more Vehicles are part of people's life in modern society, into which more and more high-tech devices are integrated, and a common platform for inter-vehicle commu- nication is necessary to realize an intelligent transportation system supporting safe driving, dynamic route scheduling, emergency message dissemination, and traffic condition monitoring. TrafficView, which is a part of the e-Road project, defines a framework to

Download

MixPad

Proceedings of the 13th international conference on Ubiquitous computing - UbiComp '11, 2011

This demo shows an interactive paper system called MixPad, which features using mice and keyboard... more This demo shows an interactive paper system called MixPad, which features using mice and keyboards to enhance the conventional pen-finger-gesture based interaction with paper documents. Similar to many interactive paper systems, MixPad adopts a mobile camera-projector unit to recognize paper documents, detect pen and finger gestures and provide visual feedback. Unlike these systems, MixPad allows using mice and keyboards to help users interact with fine-grained document content on paper (e.g. individual words and user-defined arbitrary regions), and to facilitate cross-media operations. For instance, to copy a document segment from paper to a laptop, one first points a finger of her non-dominant hand to the segment roughly, and then uses a mouse in her dominant hand to refine the selection and drag it to the laptop; she can also type text as a detailed comment on a paper document. This novel interaction paradigm combines the advantages of mice, keyboards, pens and fingers, and therefore enables rich digital functions on paper.

Large-scale EMM identification based on geometry-constrained visual word correspondence voting

Proceedings of the 1st ACM International Conference on Multimedia Retrieval - ICMR '11, 2011

We present a large-scale Embedded Media Marker (EMM) identification system which allows users to ... more We present a large-scale Embedded Media Marker (EMM) identification system which allows users to retrieve relevant dynamic media associated with a static paper document via camera-phones. The user supplies a query image by capturing an EMM-signified patch of a paper document through a camera phone. The system recognizes the query and in turn retrieves and plays the corresponding media on the phone.

Download

Minimum correspondence sets for improving large-scale augmented paper

Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry - VRCAI '11, 2011

Augmented Paper (AP) is an important area of Augmented Reality (AR). Many AP systems rely on visu... more Augmented Paper (AP) is an important area of Augmented Reality (AR). Many AP systems rely on visual features for paper document identification. Although promising, these systems can hardly support large sets of documents (i.e. one million documents) because of the high memory and time cost in handling highdimensional features. On the other hand, general large-scale image identification techniques are not well customized to AP, costing unnecessarily more resource to achieve the identification accuracy required by AP.

Download

A tele-robot assistant for remote environment management

by Chunyuan Liao and S. Lertsithichai

2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763), 2004

Using a machine to assist remote environment management can save people's time, effort, and trave... more Using a machine to assist remote environment management can save people's time, effort, and traveling cost. This paper proposes a trainable mobile robot system, which allows people to watch a remote site through a set of cameras installed on the robot, drive the platform around, and control remote devices using mouse or pen based gestures performed in video windows. Furthermore, the robot can learn device operations when it is being used by humans. After being used for a while, the robot can automatically select device control interfaces, or launch a pre-defined operation sequence based on its sensory inputs.

Download

Pacer

Proceedings of the seventeen ACM international conference on Multimedia - MM '09, 2009

Existing cameraphone-based interactive paper systems fall short of the flexibility of GUIs, partl... more Existing cameraphone-based interactive paper systems fall short of the flexibility of GUIs, partly due to their deficient finegrained interactions, limited interaction styles and inadequate targeted document types. We present PACER, a platform for applications to interact with document details (e.g. individual words, East Asian characters, math symbols, music notes, and user-specified arbitrary image regions) of generic paper documents through a camera phone. With a see-through phone interface, a user can discover symbol recurrences in a document by pointing the phone's crosshair to a symbol within a printout. The user can also continuously move the phone over a printout for gestures to copy and email an arbitrary region, or play music notes on the printout.

Download

Multichannel video/audio acquisition for immersive conferencing

2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698), 2003

This paper presents an information-driven audiovisual signal acquisition approach. This approach ... more This paper presents an information-driven audiovisual signal acquisition approach. This approach has several advantages: users are encouraged to assist in signal acquisition; available sensors are managed based on both signal characteristics and users' suggestions. The problem formulation is consistent with many well-known empirical approaches widely used in previous systems and may provide analytical explanations to these approaches. We demonstrate the use of this approach to pan/tilt/zoom (PTZ) camera management with field data.

Download

LightBeam: interacting with augmented real-world objects in pico projections

ABSTRACT Pico projectors have lately been investigated as mobile display and interaction devices.... more ABSTRACT Pico projectors have lately been investigated as mobile display and interaction devices. We propose to use them as &#39;light beams&#39;: Everyday objects sojourning in a beam are turned into dedicated projection surfaces and tangible interaction devices. This way, our daily surroundings get populated with interactive objects, each one temporarily chartered with a dedicated sub-issue of pervasive interaction. While interaction with objects has been studied in larger, immersive projection spaces, the affordances of pico projections are fundamentally different: they have a very small, strictly limited field of projection, and they are mobile. This paper contributes the results of an exploratory field study on how people interact with everyday objects in pico projections in nomadic settings. Based upon these results, we present novel interaction techniques that leverage the limited field of projection and trade-off between digitally augmented and traditional uses of everyday objects.

LightBeam: Nomadic Pico Projector Interaction with Real World Objects

Pico projectors have lately been investigated as mobile display and interaction devices. We propo... more Pico projectors have lately been investigated as mobile display and interaction devices. We propose to use them as &amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39;light beams&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39;: Everyday objects sojourning in a beam are turned into dedicated projection surfaces and tangible interaction devices. While this has been explored for large projectors, the affordances of pico projectors are fundamentally different: they have a very small and strictly limited

ButterflyNet

Proceedings of the SIGCHI conference on Human Factors in computing systems - CHI '06, 2006

Through a study of field biology practices, we observed that biology fieldwork generates a wealth... more Through a study of field biology practices, we observed that biology fieldwork generates a wealth of heterogeneous information, requiring substantial labor to coordinate and distill. To manage this data, biologists leverage a diverse set of tools, organizing their effort in paper notebooks. These observations motivated ButterflyNet, a mobile capture and access system that integrates paper notes with digital photographs captured during field research. Through ButterflyNet, the activity of leafing through a notebook expands to browsing all associated digital photos. ButterflyNet also facilitates the transfer of captured content to spreadsheets, enabling biologists to share their work. A first-use study with 14 biologists found this system to offer rich data capture and transformation, in a manner felicitous with current practice.

Download

Toward Bimanual Interactions with Mobile Projectors on Arbitrary Surfaces

The field of personal mobile projection is advancing quick- ly and a variety of work focuses on e... more The field of personal mobile projection is advancing quick- ly and a variety of work focuses on enhancing physical ob- jects in the real world with dynamically projected digital artifacts. Due to technological restrictions, none of them has yet investigated, what we feel is one of the most promising research directions: the (bimanual) interaction with mobile projections on arbitrary surfaces.

LightBeam

Proceedings of the 2012 ACM annual conference extended abstracts on Human Factors in Computing Systems Extended Abstracts - CHI EA '12, 2012

Pico projectors have lately been investigated as mobile display and interaction devices. We propo... more Pico projectors have lately been investigated as mobile display and interaction devices. We propose to use them as &amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39;light beams&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39;: Everyday objects sojourning in a beam are turned into dedicated projection surfaces and tangible interaction devices. While this has been explored for large projectors, the affordances of pico projectors are fundamentally different: they have a very small and strictly limited

LightBeam

Proceedings of the 11th International Conference on Mobile and Ubiquitous Multimedia - MUM '12, 2012

ABSTRACT Pico projectors have lately been investigated as mobile display and interaction devices.... more ABSTRACT Pico projectors have lately been investigated as mobile display and interaction devices. We propose to use them as &#39;light beams&#39;: Everyday objects sojourning in a beam are turned into dedicated projection surfaces and tangible interaction devices. This way, our daily surroundings get populated with interactive objects, each one temporarily chartered with a dedicated sub-issue of pervasive interaction. While interaction with objects has been studied in larger, immersive projection spaces, the affordances of pico projections are fundamentally different: they have a very small, strictly limited field of projection, and they are mobile. This paper contributes the results of an exploratory field study on how people interact with everyday objects in pico projections in nomadic settings. Based upon these results, we present novel interaction techniques that leverage the limited field of projection and trade-off between digitally augmented and traditional uses of everyday objects.

Embedded media barcode links

International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction on - ICMI-MLMI '10, 2010

Download

A tool for authoring unambiguous links from printed content to digital media

by Lynn Wilcox and Chunyuan Liao

Proceedings of the 19th ACM international conference on Multimedia - MM '11, 2011

Embedded Media Markers (EMMs) are nearly transparent icons printed on paper documents that link t... more Embedded Media Markers (EMMs) are nearly transparent icons printed on paper documents that link to associated digital media. By using the document content for retrieval, EMMs are less visually intrusive than barcodes and other glyphs while still providing an indication for the presence of links. An initial implementation demonstrated good overall performance but exposed difficulties in guaranteeing the creation of unambiguous EMMs. We developed an EMM authoring tool that supports the interactive authoring of EMMs via visualizations that show the user which areas on a page may cause recognition errors and automatic feedback that moves the authored EMM away from those areas. The authoring tool and the techniques it relies on have been applied to corpora with different visual characteristics to explore the generality of our approach.

Download

Pacer

Proceedings of the 28th international conference on Human factors in computing systems - CHI '10, 2010

PACER is a gesture-based interactive paper system that supports fine-grained paper document conte... more PACER is a gesture-based interactive paper system that supports fine-grained paper document content manipulation through the touch screen of a cameraphone. Using the phone's camera, PACER links a paper document to its digital version based on visual features. It adopts camerabased phone motion detection for embodied gestures (e.g. marquees, underlines and lassos), with which users can flexibly select and interact with document details (e.g. individual words, symbols and pixels). The touch input is incorporated to facilitate target selection at fine granularity, and to address some limitations of the embodied interaction, such as hand jitter and low input sampling rate. This hybrid interaction is coupled with other techniques such as semi-real time document tracking and loose physical-digital document registration, offering a gesturebased command system. We demonstrate the use of PACER in various scenarios including work-related reading, maps and music score playing. A preliminary user study on the design has produced encouraging user feedback, and suggested future research for better understanding of embodied vs. touch interaction and one vs. two handed interaction.

Download

A presentation authoring tool for media devices distributed environments

2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763), 2004

Many conference rooms are now equipped with multiple multi-media devices, such as plasma displays... more

PapierCraft: A System for Interactive Paper

The affordances of paper (e.g., ease of annotation and navigation) make it a fundamental tool for... more The affordances of paper (e.g., ease of annotation and navigation) make it a fundamental tool for knowledge gathering and crystallization tasks. During such tasks, users create a rich web of annotation and cross references. Un- fortunately, as paper is a static media, this web often gets trapped in the physical world. Some systems such as XLibris (33) address this problem by transferring this task in the digital realm where it is easy to capture all links created by the users. This approach is very powerful but suffers from the limitations of current tablet computers such as a limited screen space. In this paper we propose a paper-based interface to support the knowledge gathering and crystallization process. Our system considers document printouts as proxies of digital documents stored on the user's computer. Users can draw command gestures on printouts to indicate operations such a copying a document area, pasting an area previously copied, or creating a link. Upon pen synchro...

Download

PapierCraft

Proceedings of the 18th annual ACM symposium on User interface software and technology - UIST '05, 2005

Knowledge workers use paper extensively for document reviewing and note-taking due to its versati... more Knowledge workers use paper extensively for document reviewing and note-taking due to its versatility and simplicity of use. As users annotate printed documents and gather notes, they create a rich web of annotations and cross references. Unfortunately, as paper is a static media, this web often gets trapped in the physical world. While several digital solutions such as XLibris and Digital Desk have been proposed, they suffer from a small display size or onerous hardware requirements.

Download

Uploads

Papers by Chunyuan Liao

Log In