Exploring a Novel Mexican Sign Language Lexicon Video Dataset

arXiv (Cornell University), 2022

Despite tremendous progress in natural language processing using deep learning techniques in recent years, sign language production and comprehension has advanced very little. One critical barrier is the lack of largescale datasets available to the public due to the unbearable cost of labeled data generation. Efforts to provide public data for American Sign Language (ASL) comprehension have yielded two datasets, comprising more than thousand video clips. These datasets are large enough to enable a meaningful start to deep learning research on sign languages but are far too small to lead to any solution that can be practically deployed. So far, there is still no suitable dataset for ASL production. We proposed a system that can generate large scale ASL datasets for continuous ASL. It is suitable for general ASL processing and is particularly useful for ASL production. The continuous ASL dataset contains English labeled human articulations in condensed body pose data formats. To better serve the research community, we are releasing the first version of our ASL dataset, which contains 30k sentences, 416k words, a vocabulary of 18k words, in a total of 104 hours. This is the largest continuous sign language dataset published to date in terms of video duration. We also describe a system that can evolve and expand the dataset to incorporate better data processing techniques and more contents when available. It is our hope that the release of this ASL dataset and the sustainable dataset generation system to the public will propel better deep-learning research in ASL natural language processing.

The approval of the law for the recognition of Sign Languages and its subsequent development (together with the laws enacted by the regional governments and the work of universities and institutions such as CNLSE) has changed the landscape of the research activity carried out in the field of SL in Spain. In spite of these social advances, a corpus of Spanish Sign Language (LSE) has not yet been compiled. The average Sign Language corpus is traditionally composed of collections of annotated or tagged videos that contain written material aligned with the main Sign Language data. The compiling project presented here, CORALSE, proposes: 1) to collect a representative number of samples of language use; 2) to tag and transcribe the collected samples and build an online corpus; 3) to advance in the description of the grammar of LSE; 4) to provide the scientific background needed for the development of materials for educational purposes; and 5) to advance in the development of different types of LSE. 1 Introducción Desde nuestras primeras investigaciones en lengua de signos española (LSE), hemos sido

Proceedings of the 24th Conference on Computational Natural Language Learning

The paper presents the first dataset that aims to serve interdisciplinary purposes for the utility of computer vision community and sign language linguistics. To date, a majority of Sign Language Recognition (SLR) approaches focus on recognising sign language as a manual gesture recognition problem. However, signers use other articulators: facial expressions, head and body position and movement to convey linguistic information. Given the important role of non-manual markers, this paper proposes a dataset and presents a use case to stress the importance of including non-manual features to improve the recognition accuracy of signs. To the best of our knowledge no prior publicly available dataset exists that explicitly focuses on non-manual components responsible for the grammar of sign languages. To this end, the proposed dataset contains 28250 videos of signs of high resolution and quality, with annotation of manual and nonmanual components. We conducted a series of evaluations in order to investigate whether non-manual components would improve signs' recognition accuracy. We release the dataset to encourage SLR researchers and help advance current progress in this area toward realtime sign language interpretation. Our dataset will be made publicly available at https:// krslproject.github.io/krsl-corpus

2020

This paper presents LSE_UVIGO, a multi-source database designed to foster research on Sign Language Recognition. It is being recorded and compiled for Spanish Sign Language (LSE acronym in Spanish) and contains also spoken Galician language, so it is very well fitted to research on these languages, but also quite useful for fundamental research in any other sign language. LSE_UVIGO is composed of two datasets: LSE_Lex40_UVIGO, a multi-sensor and multi-signer dataset acquired from scratch, designed as an incremental dataset, both in complexity of the visual content and in the variety of signers. It contains static and co-articulated sign recordings, fingerspelled and gloss-based isolated words, and sentences. Its acquisition is done in a controlled lab environment in order to obtain good quality videos with sharp video frames and RGB and depth information, making them suitable to try different approaches to automatic recognition. The second subset, LSE_TVGWeather_UVIGO is being popul...

arXiv (Cornell University), 2021

In this work, we introduce the BBC-Oxford British Sign Language (BOBSL) dataset, a large-scale video collection of British Sign Language (BSL). BOBSL is an extended and publicly released dataset based on the BSL-1K dataset [1] introduced in previous work. We describe the motivation for the dataset, together with statistics and available annotations. We conduct experiments to provide baselines for the tasks of sign recognition, sign language alignment, and sign language translation. Finally, we describe several strengths and limitations of the data from the perspectives of machine learning and linguistics, note sources of bias present in the dataset, and discuss potential applications of BOBSL in the context of sign language technology. The dataset is available at ∼ vgg/data/bobsl/.

Deaf and hard-of-hearing people utilize sign language to communicate. The people around mute and deaf people, especially their family members, such as the children of deaf adults, have trouble communicating with each other if they do not know sign language. This issue has caused many researchers to conduct relevant studies concerning sign language translation or recognition; however, no public dataset is available for Malaysian words sign language. This article presents an image dataset of Malaysian Sign Language (MySL) hand gestures used in everyday situations. The contribution is a dataset for Malaysian Sign Language (MySL), which is freely accessible to all researchers and is made public. The dataset, called MyWSL2023, contains 3500 images for the ten static Malaysian sign language words collected from 5 participants (including two males and three females) in the age group of 20 to 21 years, in an indoor environment with normal lighting conditions. The dataset will allow the researchers to investigate and developed automated systems for hearing-impaired and deaf people, as well as vision-based hand gesture and sign language recognition. It is possible to access the dataset for free at https://data.mendeley.com/datasets/zvk55p7ktd.

There is a definite breakdown in communication between the hearing and Deaf communities. While automatic sign language translation (ASLT) research occurs around the world, this is not possible in South Africa due to the lack of digital South African Sign Language (SASL) gesture databases. Research into ASLT could significantly improve the day to day lives of the Deaf community by bridging the communication gap. This paper reports the development of a static SASL gesture database using a low-cost data glove. This database contains 31 static gestures, captured from five participants. These gestures have been captured in three scenarios ranging from ideal data points (Type 1) to more continuous real-world scenarios (Type 2 and 3). The overall sensor velocities were used for feature extraction when considering the continuous data in order to attain the gesture data for the neural network. This database’s effectiveness was evaluated by using neural networks to determine how accurate the classification results would be. The Levenberg-Marquardt training algorithm was used and achieved a classification accuracy of 99.61% when using the ideal (Type 1) data. The Type 2 and 3 data did not achieve a classification accuracy as high, 77.42% and 81.72% respectively when considering only 1 participant. The classification accuracy drops as more participants are evaluated using the neural network.

Behavior research methods, 2015

The LSE-Sign database is a free online tool for selecting Spanish Sign Language stimulus materials to be used in experiments. It contains 2,400 individual signs taken from a recent standardized LSE dictionary, and a further 2,700 related nonsigns. Each entry is coded for a wide range of grammatical, phonological, and articulatory information, including handshape, location, movement, and non-manual elements. The database is accessible via a graphically based search facility which is highly flexible both in terms of the search options available and the way the results are displayed. LSE-Sign is available at the following website: http://www.bcbl.eu/databases/lse/ .

2021

Computational sign language research lacks the large-scale datasets that enables the creation of useful reallife applications. To date, most research has been limited to prototype systems on small domains of discourse, e.g. weather forecasts. To address this issue and to push the field forward, we release six datasets comprised of 190 hours of footage on the larger domain of news. From this, 20 hours of footage have been annotated by Deaf experts and interpreters and is made publicly available for research purposes. In this paper, we share the dataset collection process and tools developed to enable the alignment of sign language video and subtitles, as well as baseline translation results to underpin future research.

2008

Adding American Sign Language (ASL) animation to websites can improve information access for people who are deaf with low levels of English literacy. Given a script representing the sequence of ASL signs, we must generate an animation, but a challenge is selecting accurate speed and timing for the resulting animation. In this work, we analyzed motion-capture data recorded from human ASL signers to model the realistic timing of ASL movements, with a focus on where to insert prosodic breaks (pauses), based on the sentence syntax and other features. Our methodology includes extracting data from a pre-existing ASL corpus at our lab, selecting suitable features, and building machine learning models to predict where to insert pauses. We evaluated our model using cross-validation and compared various subsets of features. Our model had 80% accuracy at predicting pause locations, out-performing a baseline model on this task.