Polar Sine Based Siamese Neural Network for Gesture Recognition

Computation

Gesture recognition is a topic in computer science and language technology that aims to interpret human gestures with computer programs and many different algorithms. It can be seen as the way computers can understand human body language. Today, the main interaction tools between computers and humans are still the keyboard and mouse. Gesture recognition can be used as a tool for communication with the machine and interaction without any mechanical device such as a keyboard or mouse. In this paper, we present the results of a comparison of eight different machine learning (ML) classifiers in the task of human hand gesture recognition and classification to explore how to efficiently implement one or more tested ML algorithms on an 8-bit AVR microcontroller for on-line human gesture recognition with the intention to gesturally control the mobile robot. The 8-bit AVR microcontrollers are still widely used in the industry, but due to their lack of computational power and limited memory, ...

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019

The version in the Kent Academic Repository may differ from the final published version. Users are advised to check http://kar.kent.ac.uk for the status of the paper. Users should always cite the published version of record.

2006 18th IEEE International Conference on Tools with Artificial Intelligence (ICTAI'06), 2006

In this paper we introduce a novel technique for modeling and recognizing gesture signals in 2D space. This technique is based on measuring the direction of the gradient of the movement trajectory as features of the gesture signal. Each gesture signal is represented as a time series of gradient angle values. These features are classified by applying a given classification method. In this article we compared the accuracy of a feed forward Artificial Neural Network with a Support Vector Machine using a radial kernel. The comparison was based on the recorded data of 13 gesture signals as training and testing data. The average accuracy of the ANN and SVM were 98.27% and 96.34% respectively. The false detection ratio was 3.83% for ANN and 8.45% for SVM, which suggests the ANN is more suitable for gesture signal recognition.

IEEE Access

The speech and hearing-impaired community use sign language as the primary means of communication. It is quite challenging for the general population to interpret or learn sign language completely. A sign language recognition system must be designed and developed to address this communication barrier. Most current sign language recognition systems rely on wearable sensors, keeping the recognition system unaffordable for most individuals. Moreover, the existing vision-based sign recognition frameworks do not consider all of the spatial and temporal information required for accurate recognition. A novel vison-based hybrid deep neural net methodology is proposed in this study for recognizing Indian and Russian sign gestures. The proposed framework is aimed to establish a single framework for tracking and extracting multisemantic properties, such as non-manual components and manual co-articulations. Furthermore, spatial feature extraction from the sign gestures is deployed using a 3D deep neural net with atrous convolutions. The temporal and sequential feature extraction is carried out by employing attention-based Bi-LSTM. In addition, the distinguished abstract feature extraction is done using the modified autoencoders. The discriminative feature extraction for differentiating the sign gestures from unwanted transition gestures is done by leveraging the hybrid attention module. The experimentation of the proposed model has been carried out on the novel multi-signer Indo-Russian sign language dataset. The proposed sign language recognition framework with hybrid neural net yields better results than other state-of-the-art frameworks.

The appropriate selection of feature extraction method plays an important role in designing a pattern recognition system. The proper representation of features contributes to a significant improvement of the classifier performance and also to the reduction of time processing. The purpose of this study is to present a description of hand's posture features based on boundary histograms. The use of histograms aims to deal with two problems: the chain small magnitude circular-shift problem caused by posture rotation and, to attenuate the non-linearity caused by shape differences when performing gesture postures. We also present a fast search start point algorithm for the boundary chain that gives a rotation invariance property to the system. The performance was evaluated using 26 postures of American Sign Language, and a comparison with other algorithms is presented. As result we obtained a robust method to be used in largescale applications using neural networks.

2019 16th International Conference on Machine Vision Applications (MVA)

We propose an efficient gesture recognition method for continuous finger gestures in untrimmed videos. We aim to discriminate similar finger gestures such as flicking. This type of gestures which are conducted only by the orientation and movement of the fingers tends to be similar, making them difficult for a correct classification since a clear temporal boundary of each target gesture is ambiguous. Thus, the recognition should focus on the accuracy to find the temporal boundaries of the target gestures. We proposed a framework based on a triplet-loss network which learns to decrease the distance of true positive boundaries while increasing that of false positive ones. Finally, we adopt a temporal representation of the segmented gesture using a stack of feature maps for gesture classification. Real-time processing and high performance are achieved with relatively compact deep learning models, which are evaluated on a new dataset of vehicle driver finger gestures. Our approach outperforms the results of previous works for online temporal segmentation and gesture classification, and it can run in real-time at 53 fps.

IEEE Signal Processing Letters, 2018

The objective of this letter is to design a unique spatiotemporal feature map characterization for three-dimensional (3-D) sign (or action) data. Current maps characterize geometric features, such as joint distances and angles or both, which could not accurately model the relative joint variations in a 3-D sign (or action) location data. Therefore, we propose a new color-coded feature map called joint angular velocity maps to accurately model the 3-D joint motions. Instead of using traditional convolutional neural networks (CNNs), we propose to develop a new ResNet architecture called connived feature ResNet, which has a CNN layer in the feedforward loop of the densely connected standard ResNet architecture. We show that this architecture avoids using dropout in the last layers and achieves the desired goal in less number of iterations compared to other ResNet and CNN based architectures used for sign (action) classification. To test our proposed model, we use our own motion captured 3-D sign language data (BVC3DSL) and other publicly available skeletal action data: CMU, HDM05, and NTU RGBD.

2008

The paper describes a method for extracting human action semantics in video's using queries-by-example.

In this research, we analyze a Convolutional Long Short-Term Memory Recurrent Neural Network (CNNL-STM) in the context of gesture recognition. CNNLSTMs are able to successfully learn gestures of varying duration and complexity. For this reason, we analyze the architecture by presenting a qualitative evaluation of the model, based on the visualization of the internal representations of the convolutional layers and on the examination of the temporal classification outputs at a frame level, in order to check if they match the cognitive perception of a gesture. We show that CNNLSTM learns the temporal evolution of the gestures classifying correctly their meaningful part, known as Kendon's stroke phase. With the visualiza-tion, for which we use the deconvolution process that maps specific feature map activations to original image pixels, we show that the network learns to detect the most intense body motion. Finally, we show that CNNLSTM outperforms both plain CNN and LSTM in gesture recognition.

2015 Data Compression Conference, 2015

This paper concerns itself with compression strategies for orientation signals, seen as signals evolving on the space of quaternions. The compression techniques extend classical signal approximation strategies used in data mining, by explicitly taking into account the quotientspace properties of the quaternion space. The approximation techniques are applied to the case of human gesture recognition from cellphone-based orientation sensors. Results indicate that the proposed approach results in high recognition accuracies, with low storage requirements, with the geometric computations providing added robustness than classical vector-space computations.