Sign Language Notation

Sign languages that exist around the world are usually identified by the country where they are used such as Ethiopian sign language. Mostly, the communication among the hearing-impaired people involves signs that stand for words by themselves. However, to make a sign language complete as a spoken language, the hearing-impaired community around the world use manual alphabets for names, technical terms, and sometimes for emphasis. As there are different alphabets for different spoken languages such as Amharic, there are manual alphabets or finger spellings used by the deaf people. Therefore, sign language in general is a tool that deaf communities use to communicate with each other.
There is no problem when the communication is limited between the deaf, but they struggle to communicate with hearing people due to the language barrier. Using translators was the solution for filling the communication gap especially in Ethiopia, even if it has its own draw backs with respect to economy or privacy issue. Consequently, developing software which fills the communication gap between the deaf and hearing people is a best solution.
This thesis contributes on the development of a model and system for Amharic sentence to Ethiopian sign language translator which accepts Amharic sentences, letters, or numbers, and outputs 3D animation of Ethiopian sign language based on the pre-lingual deaf grammar. The model bases on rule-based machine translation approaches and the developed system has three basic components: the interface component, the back-end component, and the database component. The first component (front-end) acts as a bridge between the users and the back-end component. The back-end component has three modules: Amharic text analysis, natural language processing (NLP), and text-to-sign mapping. Amharic text analysis modules analyze Amharic sentence and pass Romanized sentence to the NLP module. The NLP module accepts the Romanized Amharic sentence and performs all language processing and return sentence in Ethiopian Sign Language with including of morphological information. Then the final module (text-to-sign mapping) maps each word with the SiGML (sign script) and send to the interface component and the 3D avatar animation display the sign. In addition to enhance the quality of the translator we use a POS tagging which combine the previous work (naïve Byes classifier) and the new created one, using a brill tagging approach.
The translator performance evaluated into three classes; at sentences level, letter level, and number level and the result ranked into three categories; number of correctly translated sentences, number of understandable sentences, and number of wrong translations. All results without any errors were considered as correctly translated sentences. The results that conveyed meaning but not clear sense was considered as understandable sentences. But the results that did not covey meaning as well as sense were considered as wrong translations. Finally, the system gave an accuracy of 58.77%, 75.76%, and 84% at sentence, letter, and number level respectively.

The paper discusses how distributed logic programming can be used to define and control the hand gestures of embodied agents in virtual worlds, by using the STEP language as an interface between the constructs of logic programming and the humanoid model defined in VRML. By using this framework, different gesture dictionaries can be defined and variants of a hand gesture, according to dynamically changing factors, can be generated on the fly. The approach is tested on the demanding demonstrator of conducting, providing experience, also on time performance of the animation.

We outline the main features of our synthetic virtual human sign language system, JASigning. We describe how we have extended its input notation, SiGML, to allow explicit control of performance time, and we describe our initial steps on the path to integrating virtual human sign language performance into annotation tools, where it may be compared with video depicting the corresponding real human performance.

American Sign Language (ASL) is the natural and living language of the Deaf Community in North America. In addition to hand gestures, facial expressions are a key component of communicating in ASL. We present a method for reproducing facial expressions through computer graphic animation. Directions for further research are suggested. Keywords: Animation, American Sign Language, Facial Expressions

The paper presents selected problems of visualizing animated sign language sentences in real time. The presented solution is a part of a system for translation of texts into the sign language. The animation and graphical techniques applied in the system are briefly presented, but the main problems discussed are: how to specify the sign language and how to interpret such a specification. A concise, easy-to-use Szczepankowski's gestographic notation has been adopted. It is widely used in the Polish deaf community. It has been originally intended to be used by humans; thus a part of information it holds is incomplete, inexact, in many cases highly intuitive. The automatic interpretation has to reconstruct all the information that lacks. Another group of problems we have encountered involves issues of kinematics: the motion is to be generated depending on the information that is very general. Techniques like reverse kinematics and collision detecting and avoiding have to be applied....

This paper discusses a synthesis system that generates, from a XML input representing gesture descriptors, a vector of configuration parameters that are executed by a 3D Avatar for use in the animation of Sign Languages. The development of virtual agents able to reproduce gestures of sign languages is very important to the deaf community, since in general they also have difficulties to read conventional texts. In this research project, a consistent combination of 3D editor Blender, CMarkup parser and graphics engine Irrlicht was used to develop a novel approach to sign synthesis, based on a recent XML model that describes hand gestures using shape, location, movement and orientation descriptors. The described experiments validate the proposed implementation model, which constitutes a promising alternative in the area of synthesis of signals for computational applications of Sign Languages.

Universiteit van Stellenbosch. Sy navorsing fokus primêr op die metaleksikografie en woordeboek navorsing-'n gebied waarin hy op nasionale en internasionale vlak wyd publiseer. Hy is 'n redakteur van twee toonaangewende inter nasionale tydskrifte in die leksikografie, te wete Lexikos en Lexico graphica, asook van die boekreeks Lexicographica Series Maior. Gouws is 'n NNSgeëvalueerde navorser en die ontvanger van verskeie beurse, pryse en toekennings, onder meer die SA Akademie se CJ Langenhovenprys vir Taalwetenskap. Hy vul sy werk in die teoretiese leksikografie aan met aktiewe deelname aan die leksikografiepraktyk as redakteur/ mederedakteur van verskillende woordeboeke, onder meer die bekende Afrikaanse woordeboek HAT, die internasionale Wörterbuch zur Lexiko graphie und Wörterbuchforschung/Dictionary of lexicography and dictionary research en ANNA, die vernuwende woordeboek met Nederlands en Afrikaans as taalpaar.

Sign language synthesis has seen a large increase i n pplications over the past few decades, as it represents a poten tial solution to communication problem for the deaf community. All t hat is needed is to convert a writing form (books, newspap ers, e-mails, internet pages...) or speech into sign. Most works in th s area focus on the translation of a spoken language text into a fluid signing by using machine translation (MT), while ot hers attempt to create synthetic animation from a sign language notation. We introduce in this paper a new method for the automa ic generation of signed gestures from SignWriting nota ti n using a 3D avatar. The SW notation is provided as input in an XML based format called SWML (SignWriting Markup Langua e). Such tool would help deaf readers to grasp and inte ract with the signed transcription through a more user-friendly i nterface.

The article discusses the semantic aspects of Kazakh sign language and its characteristics. Semantics, a field within linguistics, focuses on examining the meanings conveyed by expressions and combinations of signs. The author delves into the inquiry of the degree of similarity between verbal and sign languages, highlighting their fundamental distinctions. The primary objective of the research is to scrutinize the characteristics of parts of speech in the Kazakh language when expressed gesturally, along with the principles governing the translation of verbs and adverbial tenses. The article explains in detail the formulas for translating the text into sign language, based on the subject-object-predicate. Examples are given that illustrate the subject-object relationship and determine who acts as the speaker, "object" or "subject" of the utterance. It is necessary to note that for successful translation it is necessary first to understand the meaning of the sentence. The article concludes by emphasizing the importance of understanding both structural elements and contextual nuances in the fascinating world of the semantics of the Kazakh sign language. It inspires further research aimed at uncovering the complexities and exceptions that contribute to a deep understanding of linguistic nuances in this unique form of communication.

According to the National Statistics Office (2003) in the 2000 Population Census, the deaf community in the Philippines numbered to about 121,000 deaf and hard of hearing Filipinos. Deaf and hard of hearing Filipinos in these communities use the Filipino Sign Language (FSL) as the main method of manual communication. Deaf and hard of hearing children experience difficulty in developing reading and writing skills through traditional methods of teaching used primarily for hearing children. This study aims to translate an Aesop’s fable to Filipino Sign Language with the use of 3D animation resulting to a video output. The video created contains a 3D animated avatar performing the sign translations to FSL (mainly focusing on hand gestures which includes hand shape, palm orientation, location, and movement) on screen beside their English text equivalent and related images. The final output was then evaluated by FSL deaf signers. Evaluation results showed that the final output can potenti...

Chinese Sign Language (CSL) and Chinese are languages used in the Chinese mainland. As a dominant language, Chinese has great influence on all levels of CSL. CSL, as a visual sign language, is fundamentally different from Chinese in linguistic structure. Unlike English, Chinese, as a pictograph, has influence on Chinese and CSL. This study explains in detail the influence of Chinese characters on CSL at the lexical level, including many elements from Chinese, such as "仿字 fangzi" (form imitating Chinese characters), "书空 shukong" (writing in the air with the index finger), loan translation, finger spelling, and mouthing patterns. This influence is not a simple borrowing of Chinese characters, but a creative imitation and adaptation according to the needs of sign language to express meaning. After a long period of evolution, the characteristics of Chinese characters are naturally integrated into CSL loanwords, which makes the relationship between sign language and Chinese characters closer. CSL borrows a large number of Chinese words, most of which are signs to express non-core concepts. These borrowed signs are indispensable part of CSL sign language family, enriches sign language vocabulary, improves the accuracy of sign language expression, and plays a positive role in promoting the learning, work, and lives of deaf people.

Advances in sign-language recognition technology can enable users of American Sign Language (ASL) dictionaries to search for a sign, whose meaning is unknown, by submitting a video of themselves performing the sign they had encountered, based on their memory of how it appeared. However, the relationship between the performance of signrecognition technology and user satisfaction of such search interaction is unknown. In two Wizard-of-Oz experimental studies, we found that in addition to the position of the desired word in a list of results, the similarity of the other words in the results list also affected user satisfaction.

We are aiming at developing an online collaborative framework allowing Deaf individuals to author intelligible signs using a dedicated authoring interface controlling the animation of a 3D avatar. In this paper, we present the results that we have achieved so far. They mainly focus on the design of a User Interface assisted by novel input devices. In particular, we show, in two user studies, how the Leap Motion and Kinect-like devices can be used together for capturing respectively hand trajectories (position and orientation) and facial animation. We also show how these devices are integrated into a prototype that we will demonstrate during the workshop.

Many deaf people have significant reading problems. Written content, e.g. on internet pages, is therefore not fully accessible for them. Embodied agents have the potential to communicate in the native language of this cultural group: sign language. However, state-of-the-art systems have limited comprehensibility and standard evaluation methods are missing. In this paper, we present methods and discuss challenges for the creation and evaluation of a signing avatar. We extended the existing EMBR character animation system 1 with prerequisite functionality, created a gloss-based animation tool and developed a cyclic content creation workflow with the help of two deaf sign language experts. For evaluation, we introduce delta testing, a novel way of assessing comprehensibility by comparing avatars with human signers. While our system reached stateof-the-art comprehensibility in a short development time we argue that future research needs to focus on nonmanual aspects and prosody to reach the comprehensibility levels of human signers.

We describe in this paper a system for automatically synthesizing deaf signing animations from motion data captured on real deaf subjects. Moreover, we create a virtual agent endowed with expressive gestures. Our attention is focused on the expressiveness of gesture (what type of gesture: fluidity, tension, anger) and on its semantic representations. Our approach relies on a data-driven animation scheme. From motion data captured thanks to an optical system and data gloves, we try to extract relevant features of communicative ...

The resources and technologies for Sign language processing of resourceful languages are emerging, while the low-resource languages are falling behind. Kurdish is a multi-dialect language, and it is considered a low-resource language. It is spoken by approximately 30 million people in several countries, which denotes that it has a large community with hearing-impairments as well. This paper reports on a project which aims to develop the necessary data and tools to process the Sign language for Sorani as one of the spoken Kurdish dialects. We present the results of developing a dataset in HamNoSys and its corresponding SiGML form for the Kurdish Sign lexicon. We use this dataset to implement a sign-supported Kurdish tool to check the accuracy of the Sign lexicon. We tested the tool by presenting it to hearing-impaired individuals. The experiment showed that 100% of the translated letters were understandable by a hearing-impaired person. The percentages were 65% for isolated words, an...

We describe in this paper a system for automatically synthesizing deaf signing animations from motion data captured on real deaf subjects. Moreover, we create a virtual agent endowed with expressive gestures. Our attention is focused on the expressiveness of gesture (what type of gesture: fluidity, tension, anger) and on its semantic representations. Our approach relies on a data-driven animation scheme. From motion data captured thanks to an optical system and data gloves, we try to extract relevant features of communicative ...

Software for the production of sign languages is much less common than for spoken languages. Such software usually relies on 3D humanoid avatars to produce signs which, inevitably, necessitates the use of animation. One barrier to the use of popular animation tools is their complexity and steep learning curve, which can be hard to master for inexperienced users. Here, we present PE2LGP, an authoring system that features a 3D avatar that signs Portuguese Sign Language. Our Animator is designed specifically to craft sign language animations using a key frame method, and is meant to be easy to use and learn to users without animation skills. We conducted a preliminary evaluation of the Animator, where we animated seven Portuguese Sign Language sentences and asked four sign language users to evaluate their quality. This evaluation revealed that the system, in spite of its simplicity, is indeed capable of producing comprehensible messages.

Current signing avatars are often described as unnatural as they cannot accurately reproduce all the subtleties of synchronized body behaviors of a human signer. In this paper, we propose a new dynamic approach for transitions between signs, focusing on mouthing animations for Portuguese Sign Language. Although native signers preferred animations with dynamic transitions, we did not find significant differences in comprehension and perceived naturalness scores. On the other hand, we show that including mouthing behaviors improved comprehension and perceived naturalness for novice sign language learners. Results have implications in computational linguistics, humancomputer interaction, and synthetic animation of signing avatars. 1 Deaf with a capital refers to people who identify with the deaf culture and have been deaf before they started to learn a language. They are pre-lingually deaf.

This collection of papers stems from the Fourth Workshop on the Representation and Processing of Sign Languages, held in May 2010 as a satellite to the Language Resources and Evaluation Conference in Valletta, Malta. While there has been occasional attention for sign languages at the main LREC conference, the main focus there is on spoken languages in their written and spoken forms. This series of workshops, however, offers a forum for researchers focussing on sign languages. For the second time, the workshop had sign language corpora as its main topic. With more time than in 2008, however, it was possible to include different views on corpora: The more linguistic aspects of collecting, annotating and analysing corpus data on the one hand, and sign language technologies, including vision and avatar technology, making use of sign language corpora or assisting the linguistic processing of corpora. The papers at this workshop clearly identify the potentials of even closer cooperation between sign linguists and sign language engineers, and we think it is events like this that contribute a lot to a better understanding between researchers with completely different backgrounds. The contributions composing this volume are presented in alphabetical order by the first author. For the reader's convenience, an author index is provided as well. We expect slides and posters to become available some time after the workshop at http://www.sign-lang.unihamburg.de/lrec2010/programme.html. We would like to thank all members of the programme committee who helped us reviewing an unexpectedly high number of abstracts for the workshop within a very short timeframe! Finally, we would like to point the reader to the proceedings of the previous workshops that form important resources in a growing field of research:

Current signing avatars are often described as unnatural as they cannot accurately reproduce all the subtleties of synchronized body behaviors of a human signer. In this paper, we propose a new dynamic approach for transitions between signs, focusing on mouthing animations for Portuguese Sign Language. Although native signers preferred animations with dynamic transitions, we did not find significant differences in comprehension and perceived naturalness scores. On the other hand, we show that including mouthing behaviors improved comprehension and perceived naturalness for novice sign language learners. Results have implications in computational linguistics, humancomputer interaction, and synthetic animation of signing avatars. 1 Deaf with a capital refers to people who identify with the deaf culture and have been deaf before they started to learn a language. They are pre-lingually deaf.

It is approaching a half century since Bill Stokoe published his revolutionary monograph, Sign Language Structure: An Outline of the Visual Communication Systems of the American Deaf. It is rare for a work of innovative scholarship to spark a social as well as an intellectual revolution, but that is just what Stokoe's 1960 paper did. And it is indicative both of Stokoe's genius and of his commitment that he did not simply publish his groundbreaking work and then sit back to watch the revolutions unfold. He actively promoted important changes in at least three areas of social and intellectual life. First, and perhaps most important, his work, that was ultimately generally accepted as showing the signing of deaf people to be linguistic, supported significant changes in the way deaf children are educated around the globe. Second, his work led to a general rethinking of what is fundamental about human language; and, third, it helped to reenergize the moribund field of language origin studies. This truly revolutionary paper has been reprinted at least twice, in revised and original versions, since its initial release in 1960, and now, five years after Bill's death, it is good to see it once again brought before the general public.-David F. Armstrong, Gallaudet University 'the normal' which allowed the child born deaf to survive, there would grow up between the child and those around it a communicative system derived in part from the visible parts of the paralinguistic, but much more from the kinesic, communicative behavior of the culture (Trager, 'Paralanguage', SIL 13.1-12, 1958). Based on the patterns of interactive behavior peculiar to that culture, the communication of the deaf-mute and his hearing companions would develop in different ways from the normal communication of the culture. To take a hypothetical example, a shoulder shrug,

Automatic translation between the national language and sign language is a complex process similar to translation between two different foreign languages. A very important aspect is the precision of not only manual gestures but also facial expressions, which are extremely important in the overall context of a sentence. In this article, we present the problem of including facial expressions in the automation of Polish-to-Polish Sign Language (PJM) translation—this is part of an ongoing project related to a comprehensive solution allowing for the animation of manual gestures, body movements and facial expressions. Our approach explores the possibility of using action unit (AU) recognition in the automatic annotation of recordings, which in the subsequent steps will be used to train machine learning models. This paper aims to evaluate entropy in real-life translation recordings and analyze the data associated with the detected action units. Our approach has been subjected to evaluation...

Although there has been significant advances in human-machine interaction systems in recent years, cloudbased advances are not easily integrated in autonomous machines. Here we describe a toolkit that supports interactive avatar animation and modeling for human-computer interaction. The avatar toolkit utilizes cloudbased speech-to-text software that provides active listening by detecting sound and reducing noise, a cloudbased AI to generate appropriate textual responses to user queries, and a cloud-based text-to-speech generation engine to generate utterances for this text. This output is combined with a cloud-based 3D avatar animation synchronized to the spoken response. Generated text responses are embedded within an XML structure that allows for tuning the nature of the avatar animation to simulate different emotional states. An expression package controls the avatar's facial expressions. Latency is minimized and obscured through parallel processing in the cloud and an idle loop process that animates the avatar between utterances.

We describe recent extensions to the open source Learning And Reading Assistant (LARA) supporting image-based and phonetically annotated texts. We motivate the utility of these extensions both in general and specifically in relation to endangered and archaic languages, and illustrate with examples from the revived Australian language Barngarla, Icelandic Sign Language, Irish Gaelic, Old Norse manuscripts and Egyptian hieroglyphics.

The paper reports a study that aimed to determine whether character's visual style has an effect on how signing avatars are perceived by viewers. The stimuli of the study were two polygonal characters that presented two different visual styles: stylized and realistic. Each character signed four sentences. Forty-seven participants with experience in American Sign Language (ASL) viewed the animated signing clips in random order via web survey. They (1) identified the signed sentences (if recognizable), (2) rated their legibility, and (3) rated the appeal of the signing avatar. Findings show that while character's visual style does not have an effect on subjects' perceived legibility of the signs and sign recognition, it has an effect on their interest in the character. The stylized signing avatar was perceived as more appealing than the realistic one.

We present a new highly interactive program to teach American Sign Language (ASL) for K-3 mathematics by 3D animation. We aim at increasing: (1) the opportunity for deaf children to learn arithmetic via interactive media; (2) the effectiveness of (hearing) parents in teaching arithmetic to their deaf children; (3) the effectiveness of ASL K-8 mathematics teachers. Using state-of-the-art 3D computer animation technology we have created a learning tool to be used for K-8 classroom and home learning, and for closed captioning in interactive media and internet.

In the last years we have developed a platform for the realization of embodied (conversational) agents, in a distributed logic programming framework. In this paper we will present an overview of our work, by discussing the requirements that acted as our guidelines for design decisions during development, some of the applications that have served as target demonstrators for developing and testing new functionality, and the (distributed logic programming) technology which we used for the realization of the platform and the implementation of our STEP scripting language. Although the focus of our paper will primarily be our own DLP+X3D platform, we believe that our discussion along the perspectives of requirements, applications and technology might be more generally worthwhile in establishing the relative merits of the operational use of ECA-technology. At the end of this paper, we will moreover provide some hints of how to approach the experimental validation of the (possible) benefits...

Despite research having shown that signing users vary in their preferences and needs of distinct sign language (SL) parameters, current research on SL avatars lacks consideration of the UI context's options for individualization or configuration of such parameters. Our paper addresses this gap as it presents our ability-oriented online survey, in which we asked native signers about parameters they would like to configure, as well content types that should be offered for translation within travel contexts. Our results indicate that parameters with a direct influence on the understandability of the avatar are most important for individual adjustments and situations with high time-pressure are valued highly for translation within travelling contexts. Also, our study design revealed that assessing the language background of participants in the form of an example-based self-assessment can help achieve more sensitive results, emphasizing the need for adequate research approaches. • Human-centered computing → Empirical studies in accessibility.

Computer-generated three-dimensional animation holds great promise for synthesizing utterances in American Sign Language (ASL) that are not only grammatical, but believable by members of the Deaf community. Animation poses several challenges stemming from the massive amounts of data necessary to specify the movement of three-dimensional geometry, and there is no current system that facilitates the synthesis of nonmanual signals. However, the linguistics of ASL can aid in surmounting the challenge by providing structure and rules for organizing the data. This work presents a first method for representing ASL linguistic and extralinguistic processes that involve the face. Any such representation must be capable of expressing the subtle nuances of ASL. Further, it must be able to represent co-occurrences because many ASL signs require that two or more nonmanual signals be used simultaneously. In fact simultaneity of multiple nonmanual signals can occur on the same facial feature. Additionally, such a system should allow both binary and incremental nonmanual signals to display the full range of adjectival and adverbial modifiers. Validating such a representation requires both the affirmation that nonmanual signals are indeed necessary in the animation of ASL, and the evaluation of the effectiveness of the new representation in synthesizing nonmanual signals. In this study, members of the Deaf community viewed animations created with the new representation and answered iv questions concerning the influence of selected nonmanual signals on the perceived meaning of the synthesized utterances. Results reveal that, not only is the representation capable of effectively portraying nonmanual signals, but also that it can be used to combine various nonmanual signals in the synthesis of complete ASL sentences. In a study with Deaf users, participants viewing synthesized animations consistently identified the intended nonmanual signals correctly. v

Development of automatic translation between signed and spoken languages has lagged behind the development of automatic translation between spoken languages, but it is a common misperception that extending machine translation techniques to include signed languages should be a straightforward process. A contributing factor is the lack of an acceptable method for displaying sign language apart from interpreters on video. This position paper examines the challenges of displaying a signed language as a target in automatic translation, analyses the underlying causes and suggests strategies to develop display technologies that are acceptable to sign language communities.

This document describes prototype synthetic deaf signing animation software. This software effectively forms the second half of the English-text-to-synthetic-signing pipeline described in the opening chapter of ViSiCAST deliverable D5-1. It takes as input a description of a sequence of signing gestures expressed in SiGML, from which it generates a corresponding stream of avatar animation parameters, which it then uses to drive a signing avatar on-screen. ViSiCAST D4-2 SiGML Notation-Avatar Software Driver 2 of 6

The VANESSA system provides speech-driven assistance for eGovernment and other Transactions in BSL. The Tool allows speech or text input over a limited subject domain to be translated into sign language, which is then signed for clients using an avatar. The avatar animation is generated synthetically. VANESSA also enables text communication by means of computer keyboards and an interface similar to that used by internet chat room software. The system can be easily modified to translate into other spoken languages, and multiple updates to the content are possible.

This document describes prototype synthetic deaf signing animation software. This software effectively forms the second half of the English-text-to-synthetic-signing pipeline described in the opening chapter of ViSiCAST deliverable D5-1. It takes as input a description of a sequence of signing gestures expressed in SiGML, from which it generates a corresponding stream of avatar animation parameters, which it then uses to drive a signing avatar on-screen.

We are building a system that translates German train announcements of the Swiss Federal Railways (Schweizerische Bundesbahnen, SBB) into Swiss German Sign Language (Deutschschweizerische Gebärdensprache, DSGS) in real time and displays the result via an avatar. We use the JASigning system to animate the avatar. Deliverables of the projects during which JASigning was developed are the main source of documentation for the system along with notes on the website. Not all planned features have been fully implemented: some because they are used very infrequently; others because there is insuffcient linguistic research on which to base an implementation. Our team of hearing and Deaf researchers identified the avatar functionality we needed for our project. In close collaboration with the developers of JASigning, we then found workarounds for those features that were not yet directly available in the system. Our goal for this paper is to share the solutions we found, thereby providing a us...

We outline the main features of our synthetic virtual human sign language system, JASigning. We describe how we have extended its input notation, SiGML, to allow explicit control of performance time, and we describe our initial steps on the path to integrating virtual human sign language performance into annotation tools, where it may be compared with video depicting the corresponding real human performance.

We present the technical specification for an avatar that is compliant with Animgen, the synthetic signing engine used at the University of East Anglia for generating deaf signing animations. The specification will include both the basic definition required for any standard animating avatar, and the additional parameters that Animgen requires to generate signing. Avatars compatible with Animgen are created using the ARPToolkit, an application developed at UEA that has a plug-in architecture for tools that are used for rigging an avatar mesh for animation. The toolkit also generates the additional data needed by Animgen for each avatar.

Development of automatic translation between signed and spoken languages has lagged behind the development of automatic translation between spoken languages, but it is a common misperception that extending machine translation techniques to include signed languages should be a straightforward process. A contributing factor is the lack of an acceptable method for displaying sign language apart from interpreters on video. This position paper examines the challenges of displaying a signed language as a target in automatic translation, analyses the underlying causes and suggests strategies to develop display technologies that are acceptable to sign language communities.

Creating legible animations of sign language that satisfy the needs of native users requires drawing from the fields of computational linguistics, computer animation and user experience research. This paper explores the problem of layering the many linguistic processes that contribute to a sign language utterance in avatar animation. A new framework is presented that satisfies requirements from each of these fields, and yields a flexible architecture for sign language avatars capable of leveraging a wide range of animation techniques to generate rich multi-layered animations of sign.

Without a commonly accepted writing system for American Sign Language (ASL), Deaf or Hard of Hearing (DHH) ASL signers who wish to express opinions or ask questions online must post a video of their signing, if they prefer not to use written English, a language in which they may feel less proficient. Since the face conveys essential linguistic meaning, the face cannot simply be removed from the video in order to preserve anonymity. Thus, DHH ASL signers cannot easily discuss sensitive, personal, or controversial topics in their primary language, limiting engagement in online debate or inquiries about health or legal issues. We explored several recent attempts to address this problem through development of "face swap" technologies to automatically disguise the face in videos while preserving essential facial expressions and natural human appearance. We presented several prototypes to DHH ASL signers (N=16) and examined their interests in and requirements for such technology. After viewing transformed videos of other signers and of themselves, participants evaluated the understandability, naturalness of appearance, and degree of anonymity protection of these technologies. Our study revealed users' perception of key trade-offs among these three dimensions, factors that contribute to each, and their views on transformation options enabled by this technology, for use in various contexts. Our findings guide future designers of this technology and inform selection of applications and design features. CCS CONCEPTS • Human-centered computing → Accessibility design and evaluation methods; Empirical studies in accessibility.

We are aiming at developing an online collaborative framework allowing Deaf individuals to author intelligible signs using a dedicated authoring interface controlling the animation of a 3D avatar. In this paper, we present the results that we have achieved so far. They mainly focus on the design of a User Interface assisted by novel input devices. In particular, we show, in two user studies, how the Leap Motion and Kinect-like devices can be used together for capturing respectively hand trajectories (position and orientation) and facial animation. We also show how these devices are integrated into a prototype that we will demonstrate during the workshop.

Automatic translation between the national language and sign language is a complex process similar to translation between two different foreign languages. A very important aspect is the precision of not only manual gestures but also facial expressions, which are extremely important in the overall context of a sentence. In this article, we present the problem of including facial expressions in the automation of Polish-to-Polish Sign Language (PJM) translation—this is part of an ongoing project related to a comprehensive solution allowing for the animation of manual gestures, body movements and facial expressions. Our approach explores the possibility of using action unit (AU) recognition in the automatic annotation of recordings, which in the subsequent steps will be used to train machine learning models. This paper aims to evaluate entropy in real-life translation recordings and analyze the data associated with the detected action units. Our approach has been subjected to evaluation...