Ink as Multimedia Data

1994

Abstract Pen-based computers and personal digital assistant's (PDA's) are new technologies that are growing in importance. In previous papers, we have espoused a philosophy we call "Computing in the Ink Domain" that treats ink as a first-class datatype. One of the most ...

ACM Computing Surveys, 1995

With the growing acceptance of pen-based computers and handheld personal digital assistants (PDAs), the ability to handle handwritten data is becoming an impor-tant issue. In such an environment, writ-ing, drawing, and gesturing with a stylus become the primary means of ...

2003

PDAs and notebook PCs is creating large amounts of handwritten data that need to be archived and retrieved efficiently. Word-spotting, which is based on a direct comparison of a handwritten keyword to words in the document, is commonly used for indexing and retrieval. We propose a string matching-based method for word-spotting in on-line documents. The retrieval algorithm achieves a precision of 92.3% at a recall rate of 90% on a database of 6, 672 words written by 10 different writers. Indexing experiments show an accuracy of 87.5% using a database of 3, 872 on-line words.

Electronics & Communication Engineering …, 2002

In this paper, pen computing, i.e. the use of computers and applications in which the pen is the main input device, are described from four different viewpoints. Firstly a brief overview of the hardware developments in pen systems is given, leading to the conclusion that the technological developments in this area have not led to the expected user acceptance of pen computing. The reasons underlying this market failure are explored. Problems of pen-user interface design are then described and existing and new applications are summarised. The handwriting process and product are discussed and, finally, automatic recognition methodologies are considered. Four basic factors determining handwriting variation and variability are identified. A handwriting recognition approach using segmentation into velocity-based strokes is considered in somewhat more detail

The research presented in this thesis addresses a critical issue in the introduction of new tablet-PC-based educational technology in the classroom: interpretation and semantic representation of digital ink. The teaching paradigm being investigated is that of in-class exercises, which have proven beneficial to student learning. Recent advances in educational technology support such exercises by means of distributed wireless presentation systems. Such systems have proven successful, but are limited in their scope because they rely on digital ink as the communication medium between an instructor and his or her students. The work done in this thesis extends the use of such systems and makes the following contributions towards the creation of a learning partner in the classroom:

2004

Citeseer, 2003

Calligraphic interfaces mimic the interaction of users with pen and paper, to which most persons are used to from an early age. This makes them a privileged way of interacting with computers in efficient and natural ways. One of the tasks more often made with pen and paper is to write and correct text documents. This is done resorting to several symbols with specific meanings, representing the changes to be made in the document. It should be possible to correct electronic texts in a similar way, using calligraphic interfaces. In this paper, we describe the CaliEdit text editor, currently being developed for PalmOS devices with that goal in mind. In its development, we followed a user-centered design approach. The task analysis phase, with the help of questionnaires, identified the most common symbols for the most common text-correcting tasks. The editor itself uses the CALI shape and gesture recognizer, ported to PalmOS, to help recognize those symbols. Several porting and implementation problems were overcome. Currently, heuristic and usability evaluation studies are underway to help validate the editor.

In this paper we motivate a new technique for automatic recognition of hand-sketched digital ink. By viewing sketched drawings as utterances in a visual language, sketch recognition can be posed as an ambiguous parsing problem. On this premise we have developed an algorithm for ink parsing that uses a statistical model to disambiguate. Under this formulation, writing a new recognizer for a visual language is as simple as writing a declarative grammar for the language, generating a model from the grammar, and training the model on drawing examples. We evaluate the speed and accuracy of this approach for the sample domain of the SILK visual language and report positive initial results.

Lecture Notes in Computer Science, 2018

Large and important parts of cultural heritage are stored in archives that are difficult to access, even after digitization. Documents and notes are written in hard-to-read historical handwriting and are often interspersed with illustrations. Such collections are weakly structured and largely inaccessible to a wider public and scholars. Traditionally, humanities researchers treat text and images separately. This separation extends to traditional handwriting recognition systems. Many of them use a segmentation free OCR approach which only allows the resolution of homogenous manuscripts in terms of layout, style and linguistic content. This is in contrast to our infrastructure which aims to resolve heterogeneous handwritten manuscript pages in which different scripts and images are narrowly intertwined. Authors in our use case, a 17,000 page account of exploration of the Indonesian Archipelago between 1820-1850 ("Natuurkundige Commissie voor Nederlands-Indië") tried to follow a semantic way to record their knowledge and observations, however, this discipline does not exist in the handwriting script. The use of different languages, such as German, Latin, Dutch, Malay, Greek, and French makes interpretation more challenging. Our infrastructure takes the state-of-the-art word retrieval system MONK as starting point. Owing to its visual approach, MONK can handle the diversity of material we encounter in our use case and many other historical collections: text, drawings and images. By combining text and image recognition, we significantly transcend beyond the state-of-the art, and provide meaningful additions to integrated manuscript recognition. This paper describes the infrastructure and presents early results.

2007

Pen-based interfaces offer exciting opportunities in ubiquitous computing by enabling new hardware form factors and socially acceptable computing tasks. However, prevailing ink representations are not compatible across devices or even within vendors, compromising interoperability and hence true ubiquity. We propose a lossless multimodal multi-format XML ink framework called RiverInk that overcomes many of the interoperability issues faced in using platformspecific standards. This paper motivates the interoperability problems created by ubiquity, and then describes the design of RiverInk's format, APIs, and ink controls. RiverInk's framework encompasses interoperable support for both archival and streaming inkbased applications, and a range of alternatives for platforms that do not currently support ink. Three RiverInk applications were developed within the Ac-tiveCampus ubiquitous computing environment to demonstrate both the ease of adding interoperability to an ink-based application and the degree of interoperability provided. The adaptability of the framework to new platforms is discussed.