Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Related Work
Datasets
Results
Experiment Design
Real-World Application Results on Setb
Conclusion
References
All Topics
Computer Science
Artificial Intelligence

Deep Learning Human Activity Recognition

Profile image of Steven PrestwichSteven Prestwich

2019, CEUR workshop proceedings

Abstract

Human activity recognition is an area of interest in various domains such as elderly and health care, smart-buildings and surveillance, with multiple approaches to solving the problem accurately and efficiently. For many years hand-crafted features were manually extracted from raw data signals, and activities were classified using support vector machines and hidden Markov models. To further improve on this method and to extract relevant features in an automated fashion, deep learning methods have been used. The most common of these methods are Long Short-Term Memory models (LSTM), which can take the sequential nature of the data into consideration and outperform existing techniques, but which have two main pitfalls; longer training times and loss of distant pass memory. A relevantly new type of network, the Temporal Convolutional Network (TCN), overcomes these pitfalls, as it takes significantly less time to train than LSTMs and also has a greater ability to capture more of the long term dependencies than LSTMs. When paired with a Convolutional Auto-Encoder (CAE) to remove noise and reduce the complexity of the problem, our results show that both models perform equally well, achieving state-of-the-art results, but when tested for robustness on temporal data the TCN outperforms the LSTM. The results also show, for industry applications, the TCN can accurately be used for fall detection or similar events within a smart building environment.

Related papers

A Deep Learning Approach for Human Activity Recognition Project Category: Other (Time-Series Classification)
Susana Benavidez

2019

The hardware and sensors in smartphones and wearable devices are becoming more powerful and precise every year; the valuable data these sensors collect can be used for more precise Human Activity Recognition. In this paper, we explore implementing both Convolutional Neural Networks and LSTMs to classify 18 unique activities ranging from eating chips to dribbling a ball from labelled, time-series gyroscope and accelerometer data collected from a smartwatch and smartphone. All of our work is located in our public repo here, and the processed data can be accessed here.

View PDFchevron_right
An Efficient Human Activity Recognition Technique Based on Deep Learning
Fakhreddine Ababsa

Pattern Recognition and Image Analysis, 2019

In this paper, we present a new deep learning-based human activity recognition technique. First, we track and extract human body from each frame of the video stream. Next, we abstract human silhouettes and use them to create binary space-time maps (BSTMs) which summarize human activity within a defined time interval. Finally, we use convolutional neural network (CNN) to extract features from BSTMs and classify the activities. To evaluate our approach, we carried out several tests using three public datasets: Weizmann, Keck Gesture and KTH Database. Experimental results show that our technique outperforms conventional state-of-the-art methods in term of recognition accuracy and provides comparable performance against recent deep learning techniques. It's simple to implement, requires less computing power, and can be used for multi-subject activity recognition.

View PDFchevron_right
Human Activity Recognition Using Tools of Convolutional Neural Networks: A State of the Art Review, Data Sets, Challenges and Future Prospects
Fakhri Karray

2022

Human Activity Recognition (HAR) plays a significant role in the everyday life of people because of its ability to learn extensive high-level information about human activity from wearable or stationary devices. A substantial amount of research has been conducted on HAR and numerous approaches based on deep learning and machine learning have been exploited by the research community to classify human activities. The main goal of this review is to summarize recent works based on a wide range of deep neural networks architecture, namely convolutional neural networks (CNNs) for human activity recognition. The reviewed systems are clustered into four categories depending on the use of input devices like multimodal sensing devices, smartphones, radar, and vision devices. This review describes the performances, strengths, weaknesses, and the used hyperparameters of CNN architectures for each reviewed system with an overview of available public data sources. In addition, a discussion with t...

View PDFchevron_right
Human Activity Recognition Based on Deep-Temporal Learning Using Convolution Neural Networks Features and Bidirectional Gated Recurrent Unit With Features Selection
Hathal Alwageed

IEEE Access

Recurrent Neural Networks (RNNs) and their variants have been demonstrated tremendous successes in modeling sequential data such as audio processing, video processing, time series analysis, and text mining. Inspired by these facts, we propose human activity recognition technique to proceed visual data via utilizing convolution neural network (CNN) and Bidirectional-gated recurrent unit (Bi-GRU). Firstly, we extract deep features from frames sequence of human activities videos using CNN and then select most important features from the deep appearances to improve performance and decrease computational complexity of the model. Secondly, to learn temporal motions of frames sequence, we design Bi-GRU and feed those deep-important features extracted from frames sequence of human activities to Bi-GRU which learn temporal dynamics in forward and backward direction at each time step. We conduct extensive experiments on realistic videos of human activity recognition datasets YouTube11, HMDB51 and UCF101. Lastly, we compare the obtained results with existing methods to show the competence of our proposed technique. INDEX TERMS Human activity recognition, recurrent neural networks (RNNs), convolution neural networks (CNNs), bidirectional-gated recurrent unit (Bi-GRU), deep learning.

View PDFchevron_right
Long Short Term Memory network for Recognition of Daily Human Activity
Anvaya Ahlawat

2021

Previously Human Activity Recognition has been solved by using the engineered features, but the main problem with this approach is that it requires specific domain knowledge. Adopting classical machine learning models has been effective but these methods completely ignore the time signal obtained from the sensors. But due to recent advancements, deep learning techniques like Long Short Term Model and recurrent neural networks have been effectively used to provide good results and classify human activities correctly as compared to machine learning models. So in this paper, we propose a LSTM framework to classify these activities and also compare its performance with classical machine learning models.

View PDFchevron_right
Hybrid deep learning framework for human activity recognition
SHRISHAIL SIDRAMAYYA MATH

International Journal of Nonlinear Analysis and Applications, 2022

The aim of the recognition in the human activity is to recognize the actions of the individuals using a set of observations and their environmental conditions. Since last two decades, the research on this Human Activity Recognition (HAR) has captured the attention of several computer science communities because of the strength to provide support to different applications and the connection to different fields of study such as, human-computer interaction, healthcare, monitoring, entertainment and education. There are many existing methods like deep learning which have been used to develop to recognize the different activities of the human, but couldn’t identify the sudden change of the activities in the human. This paper presents a method using the deep learning methods which can recognize the specific identities and identify a change from one activity to another for the applications of the healthcare. In this method, a deep convolutional neural network is built using which the featu...

View PDFchevron_right
Performance Analysis of Deep Learning based Human Activity Recognition Methods
A F M Zainul Abadin

Journal of Applied Science & Process Engineering, 2022

Human Activity Recognition (HAR) is one of the most important branches of human-centered research activities. Along with the development of artificial intelligence, deep learning techniques have gained remarkable success in computer vision. In recent years, there is a growing interest in Human Activity Recognition systems applied in healthcare, security surveillance, and human motion-based activities. A HAR system is essentially made of a wearable device equipped with a set of sensors (like accelerometers, gyroscopes, magnetometers, heart-rate sensors, etc.). Different methods are being applied for improving the accuracy and performance of the HAR system. In this paper, we implement Artificial Neural Network (ANN), and Convolutional Neural Network (CNN) in combination with Long Short-term Memory (LSTM) methods with different layers and compare their outputs towards the accuracy in the HAR system. We compare the accuracy of different HAR methods and observed that the performance of our proposed model of CNN 2 layers with LSTM 1 layer is the best.

View PDFchevron_right
Human Activity Recognition Using Deep Learning: A Survey
Bijal Gadhia

Lecture Notes on Data Engineering and Communications Technologies

Human activity recognition refers to predict what a person is doing from series of the observation of person's action and surrounding conditions using different techniques. It is an active research area providing personalized support for various applications and its association with a wide range of fields of study like medicinal services, dependable automation developing, and smart surveillance system. This paper provides an overview which gives idea about some existing research methods on human activity recognition. It describes a general view on the state of the art for human activity recognition and shows comparative studies between existing research which consist of various methods, evaluation criteria, and features. It also comprises benefits and limitations of various methods to provide researchers to propose new approaches.

View PDFchevron_right
Human Activity Recognition Using Recurrent Neural Networks
Yoshihiro Ando

Lecture Notes in Computer Science, 2017

Human activity recognition using smart home sensors is one of the bases of ubiquitous computing in smart environments and a topic undergoing intense research in the field of ambient assisted living. The increasingly large amount of data sets calls for machine learning methods. In this paper, we introduce a deep learning model that learns to classify human activities without using any prior knowledge. For this purpose, a Long Short Term Memory (LSTM) Recurrent Neural Network was applied to three real world smart home datasets. The results of these experiments show that the proposed approach outperforms the existing ones in terms of accuracy and performance.

View PDFchevron_right
Enhanced Recognition of Human Activity using Hybrid Deep Learning Techniques
FOREX Publication, Abinaya S, Potti Sai Pavan Guru Jayanth

FOREX Publication, 2024

In the domain of deep learning, Human Activity Recognition (HAR) models stand out, surpassing conventional methods. These cutting-edge models excel in autonomously extracting vital data features and managing complex sensor data. However, the evolving nature of HAR demands costly and frequent retraining due to subjects, sensors, and sampling rate variations. To address this challenge, we introduce Cross-Domain Activities Analysis (CDAA) combined with a clustering-based Gated Recurrent Unit (GRU) model. CDAA reimagines motion clusters, merging origin and destination movements while quantifying domain disparities. Expanding our horizons, we incorporate image datasets, leveraging Convolutional Neural Networks (CNNs). The innovative aspects of the proposed hybrid GRU_CNN model, showcasing its superiority in addressing specific challenges in human activity recognition, such as subject and sensor variations. This approach consistently achieves 98.5% accuracy across image, UCI-HAR, and PAMAP2 datasets. It excels in distinguishing activities with similar postures. Our research not only pushes boundaries but also reshapes the landscape of HAR, opening doors to innovative applications in healthcare, fitness tracking, and beyond.

View PDFchevron_right

References (24)

  1. Abdelnasser, H., Youssef, M., Harras, K.A.: Wigest: A ubiquitous wifi-based ges- ture recognition system. In: IEEE INFOCOM. pp. 1472-1480 (2015)
  2. Bai, S., Kolter, J.Z., Koltun, V.: An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. CoRR abs/1803.01271 (2018)
  3. Cippitelli, E., Fioranelli, F., Gambi, E., Spinsante, S.: Radar and rgb-depth sensors for fall detection: A review. IEEE Sensors Journal 17(12), 3585-3604 (2017)
  4. Dollár, P., Rabaud, V., Cottrell, G., Belongie, S.: Behavior recognition via sparse spatio-temporal features. VS-PETS Beijing, China (2005)
  5. Halperin, D., Hu, W., Sheth, A., Wetherall, D.: Tool release: Gathering 802.11 n traces with channel state information. ACM SIGCOMM Computer Communica- tion Review 41(1), 53-53 (2011)
  6. Hou, J., Wang, G., Chen, X., Xue, J.H., Zhu, R., Yang, H.: Spatial-temporal at- tention res-tcn for skeleton-based dynamic hand gesture recognition. In: Proc. of the ECCV. pp. 0-0 (2018)
  7. Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei-Fei, L.: Large- scale video classification with convolutional neural networks. In: Proc. of the IEEE conference on CVPR. pp. 1725-1732 (2014)
  8. Laptev, I.: On space-time interest points. IJCV 64(2-3), 107-123 (2005)
  9. Lea, C., Flynn, M.D., Vidal, R., Reiter, A., Hager, G.D.: Temporal convolutional networks for action segmentation and detection. In: Proc. of the IEEE Conference on CVPR. pp. 156-165 (2017)
  10. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. nature 521(7553), 436 (2015)
  11. Li, H., Ota, K., Dong, M., Guo, M.: Learning human activities through wi-fi channel state information with multiple access points. IEEE Com Mag 56(5) (2018)
  12. Morales, J., Akopian, D.: Physical activity recognition by smartphones, a survey. Biocybernetics and Biomedical Engineering 37(3), 388-400 (2017)
  13. Nair, N., Thomas, C., Jayagopi, D.B.: Human activity recognition using temporal convolutional network. In: Proc. of the 5th iWOAR. p. 17 (2018)
  14. Pu, Q., Gupta, S., Gollakota, S., Patel, S.: Whole-home gesture recognition using wireless signals. In: Proc. of the 19th ACM MobiCom. pp. 27-38 (2013)
  15. Qolomany, B., Al-Fuqaha, A., Benhaddou, D., Gupta, A.: Role of deep lstm neural networks and wi-fi networks in support of occupancy prediction in smart buildings. In: IEEE 19th Int. Conf. on HPCC; IEEE 15th Int. Conf. on SmartCity; IEEE 3rd Int. Conf. on DSS. pp. 50-57 (2017)
  16. Ronao, C.A., Cho, S.B.: Human activity recognition with smartphone sensors using deep learning neural networks. Expert systems with apls 59, 235-244 (2016)
  17. Trȃscȃu, M., Nan, M., Florea, A.M.: Spatio-temporal features in action recognition using 3d skeletal joints. Sensors 19(2), 423 (2019)
  18. Wang, F., Gong, W., Liu, J., Wu, K.: Channel selective activity recognition with wifi: A deep learning approach exploring wideband information. IEEE Transactions on Network Science and Engineering (2018)
  19. Wang, G., Zou, Y., Zhou, Z., Wu, K., Ni, L.M.: We can hear you with wi-fi! IEEE Transactions on Mobile Computing 15(11), 2907-2920 (2016)
  20. Wang, Y., Wu, K., Ni, L.M.: Wifall: Device-free fall detection by wireless networks. IEEE Transactions on Mobile Computing 16(2), 581-594 (2016)
  21. Yatani, K., Truong, K.N.: Bodyscope: a wearable acoustic sensor for activity recog- nition. In: Proc. of ACM Conference on Ubiquitous Computing. pp. 341-350 (2012)
  22. Yousefi, S., Narui, H., Dayal, S., Ermon, S., Valaee, S.: A survey on behavior recognition using wifi channel state information. IEEE Comms Mag. 55(10) (2017)
  23. Yue-Hei Ng, J., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., Toderici, G.: Beyond short snippets: Deep networks for video classification. In: Proc. of the IEEE conference on CVPR. pp. 4694-4702 (2015)
  24. Zou, H., Zhou, Y., Yang, J., Spanos, C.J.: Towards occupant activity driven smart buildings via wifi-enabled iot devices and deep learning. Energy and Buildings 177, 12-22 (2018)

Related papers

A Close Look into Human Activity Recognition Models using Deep Learning
Naeem Seliya

2022 3rd International Conference on Computing, Networks and Internet of Things (CNIOT)

Human activity recognition using deep learning techniques has become increasing popular because of its high effectivity with recognizing complex tasks, as well as being relatively low in costs compared to more traditional machine learning techniques. This paper surveys some state-of-the-art human activity recognition models that are based on deep learning architecture and has layers containing Convolution Neural Networks (CNN), Long Short-Term Memory (LSTM), or a mix of more than one type for a hybrid system. The analysis outlines how the models are implemented to maximize its effectivity and some of the potential limitations it faces.

View PDFchevron_right
Sensor-Based Human Activity Recognition with Spatio-Temporal Deep Learning
mansour alsulaiman

Sensors, 2021

Human activity recognition (HAR) remains a challenging yet crucial problem to address in computer vision. HAR is primarily intended to be used with other technologies, such as the Internet of Things, to assist in healthcare and eldercare. With the development of deep learning, automatic high-level feature extraction has become a possibility and has been used to optimize HAR performance. Furthermore, deep-learning techniques have been applied in various fields for sensor-based HAR. This study introduces a new methodology using convolution neural networks (CNN) with varying kernel dimensions along with bi-directional long short-term memory (BiLSTM) to capture features at various resolutions. The novelty of this research lies in the effective selection of the optimal video representation and in the effective extraction of spatial and temporal features from sensor data using traditional CNN and BiLSTM. Wireless sensor data mining (WISDM) and UCI datasets are used for this proposed metho...

View PDFchevron_right
Deep Recurrent Neural Networks for Human Activity Recognition
Abdulmajid Murad

Sensors

Adopting deep learning methods for human activity recognition has been effective in extracting discriminative features from raw input sequences acquired from body-worn sensors. Although human movements are encoded in a sequence of successive samples in time, typical machine learning methods perform recognition tasks without exploiting the temporal correlations between input data samples. Convolutional neural networks (CNNs) address this issue by using convolutions across a one-dimensional temporal sequence to capture dependencies among input data. However, the size of convolutional kernels restricts the captured range of dependencies between data samples. As a result, typical models are unadaptable to a wide range of activity-recognition configurations and require fixed-length input windows. In this paper, we propose the use of deep recurrent neural networks (DRNNs) for building recognition models that are capable of capturing long-range dependencies in variable-length input sequences. We present unidirectional, bidirectional, and cascaded architectures based on long short-term memory (LSTM) DRNNs and evaluate their effectiveness on miscellaneous benchmark datasets. Experimental results show that our proposed models outperform methods employing conventional machine learning, such as support vector machine (SVM) and k-nearest neighbors (KNN). Additionally, the proposed models yield better performance than other deep learning techniques, such as deep believe networks (DBNs) and CNNs.

View PDFchevron_right
Human Activity Recognition Using CNN and Lstm Deep Learning Algorithms
geeta mara

Journal of Advanced Zoology, 2023

Human Activity Recognition recognizes and classifies the activities performed by the users or people based on the data collected from the sensors of special devices such as smart-watches, smartphones etc. It has become easy to collect a huge amount of data from inertial sensors that are embedded in wearable devices. An accelerometer and gyroscope sensors are most commonly used inertial sensors. There are various already available datasets, in our paper, we are using the Wireless Sensor Data Mining dataset which contains 1,098,207 data of 6 physical activities performed. In this paper, the activities we aim to classify are walking, jogging, going up and downstairs, standing, and sitting. There are various algorithms applied on the various datasets. In our paper, we use Convolutional Neural Network and Long Short-Term Memory deep learning algorithm on the data set, we split the data into training data [80%] and testing data [20%]. By using a confusion matrix, we recognize and classify the activities performed using maximum accuracy.

View PDFchevron_right
A Hybrid Deep Neural Network for Human Activity Recognition based on IoT Sensors
Zakaria Benhaili

International Journal of Advanced Computer Science and Applications, 2021

Internet of things (IOT) sensors, has received a lot of interest in recent years due to the rise of application demands in domains like ubiquitous and context-aware computing, activity surveillance, ambient assistive living and more specifically in Human activity recognition. The recent development in deep learning allows to extract high-level features automatically, and eliminates the reliance on traditional machine learning techniques, which depended heavily on hand crafted features. In this paper, we introduce a network that can identify a variety of everyday human actions that can be carried out in a smart home environment, by using raw signals generated from Internet of Thing's motion sensors. We design our architecture basing on a combination of convolutional neural network (CNN) and Gated recurrent unit (GRU) layers. The CNN is first deployed to extract local and scale-invariance features, then the GRU layers are used to extract sequential temporal dependencies. We tested our model called (CNGRU) on three public datasets. It achieves an accuracy better or comparable to existing state of the art models.

View PDFchevron_right

Related topics

  • Computer Science
  • Artificial Intelligence
  • Fall detection
  • Long Short Term Memory
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved