Human activity recognition for domestic robots

2017

Human activity recognition (HAR) has gained an effective role for computer vision in the problem of video surveillance systems and medical purposes. This study represents an approach for a biometric system that can recognize human activities in 3D space. Related approaches have shown that monitoring of various physiological signals can provide distinctive information to recognize human activities. The proposed method conducts a machine learning to determine a pattern on numerous activities using angles between joints in 3D space. Basically, activity related joint angles are obtained using Microsoft Kinect v2 sensor. Since HAR is operated in a time domain, joints' angle information is stored in every 20 frames. Stored information consist the feature set for human activity recognition. Support vector machine (SVM) is used for both train and test. Among six different human activities, obtained average accuracy is 99,999482 %.

2013 IEEE 8th Conference on Industrial Electronics and Applications (ICIEA)

Ability to recognize human activities will enhance the capabilities of a robot that interacts with humans. However automatic detection of human activities could be challenging due to the individual nature of the activities. In this paper, we present human activity detection model that uses only 3-D skeleton features generated from an RGB-D sensor (Microsoft KinectTM). To infer the human activities, we implemented Gaussian Mixture Modal (GMM) based Hidden Markov Model(HMM). GM outputs of the HMM were effectively able to capture multimodel nature of 3D positions of each skeleton joint. We test our model in a publicly available data-set that consists of twelve different daily activities performed by four different people. The proposed model recorded recognition recall accuracy of 84% with previously seen people and 78% with previously unseen people.

ETRI Journal, 2019

Human activity recognition (HAR) has become effective as a computer vision tool for video surveillance systems. In this paper, a novel biometric system that can detect human activities in 3D space is proposed. In order to implement HAR, joint angles obtained using an RGB-depth sensor are used as features. Because HAR is operated in the time domain, angle information is stored using the sliding kernel method. Haar-wavelet transform (HWT) is applied to preserve the information of the features before reducing the data dimension. Dimension reduction using an averaging algorithm is also applied to decrease the computational cost, which provides faster performance while maintaining high accuracy. Before the classification, a proposed thresholding method with inverse HWT is conducted to extract the final feature set. Finally, the K-nearest neighbor (k-NN) algorithm is used to recognize the activity with respect to the given data. The method compares favorably with the results using other machine learning algorithms.

2012 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), 2012

We address the problem of classifying human actions using a single depth sensor camera. In this work, we propose an angular representation to model the relationship between the joints in human skeleton. This representation helps cope with noisy data while enhances both computational efficiency and flexibility. Also, we propose to use Hidden Markov Model (HMM) to recognize temporal motion patterns. The full skeleton formulated in a 60D feature vector is tuned to a 37D feature vector of the most active joints. These features are then fed to the HMM for recognition. We evaluate our classifier on a dataset of 19 classes and 5 indoor scenarios with hundreds of action instances recorded using the Microsoft XBOX Kinect J sensor and achieve an average precision/recall of 91.14%/96.89%.

2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016

Social activity based on body motion is a key feature for non-verbal and physical behavior defined as function for communicative signal and social interaction between individuals. Social activity recognition is important to study human-human communication and also human-robot interaction. Based on that, this research has threefold goals: (1) recognition of social behavior (e.g. human-human interaction) using a probabilistic approach that merges spatio-temporal features from individual bodies and social features from the relationship between two individuals; (2) learn priors based on physical proximity between individuals during an interaction using proxemics theory to feed a probabilistic ensemble of activity classifiers; and (3) provide a public dataset with RGB-D data of social daily activities including risk situations useful to test approaches for assisted living, since this type of dataset is still missing. Results show that using the proposed approach designed to merge features with different semantics and proximity priors improves the classification performance in terms of precision, recall and accuracy when compared with other approaches that employ alternative strategies.

IET Computer Vision

Human activity recognition is an important and active field of research having a wide range of applications in numerous fields including ambient-assisted living (AL). Although most of the researches are focused on the single user, the ability to recognise two-person interactions is perhaps more important for its social implications. This study presents a twoperson activity recognition system that uses skeleton data extracted from a depth camera. The human actions are encoded using a set of a few basic postures obtained with an unsupervised clustering approach. Multiclass support vector machines are used to build models on the training set, whereas the X-means algorithm is employed to dynamically find the optimal number of clusters for each sample during the classification phase. The system is evaluated on the Institute of Systems and Robotics (ISR)-University of Lincoln (UoL) and Stony Brook University (SBU) datasets, reaching overall accuracies of 0.87 and 0.88, respectively. Although the results show that the performances of the system are comparable with the state of the art, recognition improvements are obtained with the activities related to health-care environments, showing promise for applications in the AL realm.

Proceedings of the AsiaPacific Signal and Information Processing Association Annual Summit and Conference, 2014

Human activity recognition is crucial for intuitive cooperation between humans and robots. We present an approach for activity recognition for applications in the context of human-robot interaction in industrial settings. The approach is based on spatial and temporal features derived from skeletal data of human workers performing assembly tasks. These features were used to train a machine learning framework, which classifies discrete time frames with Random Forests and subsequently models temporal dependencies between the resulting states with a Hidden Markov Model. We considered the following three groups of activities: Movement, Gestures, and Object handling. A dataset has been collected which is comprised of 24 recordings of several human workers performing such activities in a human-robot interaction environment, as typically seen at small and medium-sized enterprises. The evaluation shows that the approach achieves a recognition accuracy of up to 88% for some activities and an average accuracy of 73%.

The 7th International Conference on Information Technology, 2015

The analysis of human activities is one of the most interesting and important open issues for the automated video surveillance community. In order to understand the behaviors of humans, a higher level of understanding is required, which is generally referred to as activity recognition. While traditional approaches rely on 2D data like images or videos, the development of low-cost depth sensors created new opportunities to advance the field. In this paper, a system to recognize human activities using 3D skeleton joints recovered from 3D depth data of RGB-D cameras is proposed. A low dimensional descriptor is constructed for activity recognition based on skeleton joints. The proposed system focuses on recognizing human activities not human actions. Human activities take place over different time scales and consist of a sequence of sub-activities (referred to as actions). The proposed system recognizes learned activities via trained Hidden Markov Models (HMMs). Experimental results on two human activity recognition benchmarks show that the proposed recognition system outperforms various state-of-the-art skeleton-based human activity recognition techniques.

International Journal of Scientific & Engineering Research, 2015

There is a significantly increasing demand for monitoring systems for elderly people in the health-care sector. As the aging population increases, patient privacy violations and the cost of elderly assistance have driven the research community toward computer vision and image processing to design and deploy new systems for monitoring the elderly in the authors' society and turning their living houses into smart environments. By exploiting recent advances and the low cost of threedimensional (3D) depth sensors such as Microsoft Kinect, the authors propose a new skeleton-based approach to describe the spatio-temporal aspects of a human activity sequence, using the Minkowski and cosine distances between the 3D joints. We trained and validated their approach on the Microsoft MSR 3D Action and MSR Daily Activity 3D datasets using the Extremely Randomised Trees algorithm. The results are very promising, demonstrating that the trained model can be used to build a monitoring system for the elderly using open-source libraries and a low-cost depth sensor.

IET Computer Vision, 2017

There is a significantly increasing demand for monitoring systems for elderly people in the health-care sector. As the aging population increases, patient privacy violations and the cost of elderly assistance have driven the research community toward computer vision and image processing to design and deploy new systems for monitoring the elderly in the authors' society and turning their living houses into smart environments. By exploiting recent advances and the low cost of threedimensional (3D) depth sensors such as Microsoft Kinect, the authors propose a new skeleton-based approach to describe the spatio-temporal aspects of a human activity sequence, using the Minkowski and cosine distances between the 3D joints. We trained and validated their approach on the Microsoft MSR 3D Action and MSR Daily Activity 3D datasets using the Extremely Randomised Trees algorithm. The results are very promising, demonstrating that the trained model can be used to build a monitoring system for the elderly using open-source libraries and a low-cost depth sensor.