Academia.eduAcademia.edu

Outline

Human Gait Activity Recognition Machine Learning Methods

Sensors

https://doi.org/10.3390/S23020745

Abstract

Human gait activity recognition is an emerging field of motion analysis that can be applied in various application domains. One of the most attractive applications includes monitoring of gait disorder patients, tracking their disease progression and the modification/evaluation of drugs. This paper proposes a robust, wearable gait motion data acquisition system that allows either the classification of recorded gait data into desirable activities or the identification of common risk factors, thus enhancing the subject’s quality of life. Gait motion information was acquired using accelerometers and gyroscopes mounted on the lower limbs, where the sensors were exposed to inertial forces during gait. Additionally, leg muscle activity was measured using strain gauge sensors. As a matter of fact, we wanted to identify different gait activities within each gait recording by utilizing Machine Learning algorithms. In line with this, various Machine Learning methods were tested and compared to...

FAQs

sparkles

AI

What advanced machine learning method is most effective for gait event detection?add

The study identifies the attention-based Convolutional Neural Network and Recurrent Neural Network (CNNA + RNN) as the most reliable method for detecting specific gait events, achieving high classification metrics across varied datasets.

What unique feature does the gait motion data acquisition system offer?add

The proposed system enables semi-automatic labeling of gait data in real-time, improving the personalization and accuracy of gait analysis for individual subjects.

Which sensor placements yielded the highest performance in data collection?add

Sensor placement below the knee joint, anterior to the shin bone, resulted in the highest data acquisition performance, significantly enhancing ML classification capabilities.

How did inertial measurement units (IMUs) improve gait analysis?add

IMUs integrated into the system provided diverse data through low power, compact design, enhancing real-time accelerations measurements during various gait activities.

What challenges exist in developing universal ML algorithms for gait analysis?add

Variations in individual gait profiles and specific gait disorders like Freezing of Gait (FOG) complicate universal algorithm training, necessitating personalized ML approaches for effective detection.

References (57)

  1. Saunders, J.B.D.M.; Inman, V.T.; Eberhart, H.D. The Major Determinants in Normal and Pathological Gait. J. Bone Jt. Surg. 1953, 35, 543-558. [CrossRef]
  2. Bettecken, K.; Bernhard, F.; Sartor, J.; Hobert, M.A.; Hofmann, M.; Gladow, T.; van Uem, J.M.T.; Liepelt-Scarfone, I.; Maetzler, W. No relevant association of kinematic gait parameters with Health-related Quality of Life in Parkinson's disease. PLoS ONE 2017, 12, e0176816. [CrossRef] [PubMed]
  3. Mazzetta, I.; Zampogna, A.; Suppa, A.; Gumiero, A.; Pessione, M.; Irrera, F. Wearable Sensors System for an Improved Analysis of Freezing of Gait in Parkinson's Disease Using Electromyography and Inertial Signals. Sensors 2019, 19, 948. [CrossRef]
  4. Taborri, J.; Palermo, E.; Rossi, S.; Cappa, P. Gait Partitioning Methods: A Systematic Review. Sensors 2016, 16, 66. [CrossRef] [PubMed]
  5. Mo, L.; Zeng, L. Running gait pattern recognition based on cross-correlation analysis of single acceleration sensor. Math. Biosci. Eng. 2019, 16, 6242-6256. [CrossRef]
  6. DeJong, M.A.F.; Hertel, J. Validation of Foot-Strike Assessment Using Wearable Sensors During Running. J. Athl. Train. 2020, 55, 1307-1310. [CrossRef]
  7. Benson, L.C.; Clermont, C.A.; Bošnjak, E.; Ferber, R. The use of wearable devices for walking and running gait analysis outside of the lab: A systematic review. Gait Posture 2018, 63, 124-138. [CrossRef]
  8. Goršič, M.; Kamnik, R.; Ambrožič, L.; Vitiello, N.; Lefeber, D.; Pasquini, G.; Munih, M. Online Phase Detection Using Wearable Sensors for Walking with a Robotic Prosthesis. Sensors 2014, 14, 2776-2794. [CrossRef]
  9. Bejarano, N.C.; Ambrosini, E.; Pedrocchi, A.; Ferrigno, G.; Monticone, M.; Ferrante, S. A Novel Adaptive, Real-Time Algorithm to Detect Gait Events from Wearable Sensors. IEEE Trans. Neural Syst. Rehabil. Eng. 2015, 23, 413-422. [CrossRef]
  10. Jiang, X.; Chu, K.H.; Khoshnam, M.; Menon, C. A Wearable Gait Phase Detection System Based on Force Myography Techniques. Sensors 2018, 18, 1279. [CrossRef]
  11. Seel, T.; Raisch, J.; Schauer, T. IMU-Based Joint Angle Measurement for Gait Analysis. Sensors 2014, 14, 6891-6909. [CrossRef] [PubMed]
  12. Benson, L.C.; Clermont, C.A.; Watari, R.; Exley, T.; Ferber, R. Automated Accelerometer-Based Gait Event Detection during Multiple Running Conditions. Sensors 2019, 19, 1483. [CrossRef] [PubMed]
  13. Su, B.; Smith, C.; Farewik, E.G. Gait Phase Recognition Using Deep Convolutional Neural Network with Inertial Measurement Units. Biosensors 2020, 10, 109. [CrossRef]
  14. Moustakidis, S.P.; Theocharis, J.B.; Giakas, G. Subject Recognition Based on Ground Reaction Force Measurements of Gait Signals. IEEE Trans. Syst. Man Cybern. Part B (Cybernetics) 2008, 38, 1476-1485. [CrossRef]
  15. Woznowski, P.; Kaleshi, D.; Oikonomou, G.; Craddock, I. Classification and suitability of sensing technologies for activity recognition. Comput. Commun. 2016, 89-90, 34-50. [CrossRef]
  16. Aminian, K.; Najafi, B.; Büla, C.; Leyvraz, P.-F.; Robert, P. Spatio-temporal parameters of gait measured by an ambulatory system using miniature gyroscopes. J. Biomech. 2002, 35, 689-699. [CrossRef] [PubMed]
  17. Taborri, J.; Rossi, S.; Palermo, E.; Patanè, F.; Cappa, P. A Novel HMM Distributed Classifier for the Detection of Gait Phases by Means of a Wearable Inertial Sensor Network. Sensors 2014, 14, 16212-16234. [CrossRef]
  18. Sprager, S.; Juric, M.B. Inertial Sensor-Based Gait Recognition: A Review. Sensors 2015, 15, 22089-22127. [CrossRef]
  19. Benson, L.C.; Räisänen, A.M.; Clermont, C.A.; Ferber, R. Is This the Real Life, or Is This Just Laboratory? A Scoping Review of IMU-Based Running Gait Analysis. Sensors 2022, 22, 1722. [CrossRef]
  20. Lopez-Nava, I.H.; Garcia-Constantino, M.; Favela, J. Recognition of Gait Activities Using Acceleration Data from A Smartphone and A Wearable Device. Proceedings 2019, 31, 60. [CrossRef]
  21. Takeda, R.; Tadano, S.; Natorigawa, A.; Todoh, M.; Yoshinari, S. Gait posture estimation using wearable acceleration and gyro sensors. J. Biomech. 2009, 42, 2486-2494. [CrossRef] [PubMed]
  22. Prado, A.; Cao, X.; Robert, M.T.; Gordon, A.; Agrawal, S.K. Gait Segmentation of Data Collected by Instrumented Shoes Using a Recurrent Neural Network Classifier. Phys. Med. Rehabil. Clin. N. Am. 2019, 30, 355-366. [CrossRef] [PubMed]
  23. Hebenstreit, F.; Leibold, A.; Krinner, S.; Welsch, G.; Lochmann, M.; Eskofier, B.M. Effect of walking speed on gait sub phase durations. Hum. Mov. Sci. 2015, 43, 118-124. [CrossRef]
  24. Kawabata, M.; Goto, K.; Fukusaki, C.; Sasaki, K.; Hihara, E.; Mizushina, T.; Ishii, N. Acceleration patterns in the lower and upper trunk during running. J. Sports Sci. 2013, 31, 1841-1853. [CrossRef] [PubMed]
  25. Pappas, I.P.I.; Popovic, M.R.; Keller, T.; Dietz, V.; Morari, M. A reliable gait phase detection system. IEEE Trans. Neural Syst. Rehabil. Eng. 2001, 9, 113-125. [CrossRef] [PubMed]
  26. Ferreira, P.J.S.; Cardoso, J.M.P.; Mendes-Moreira, J. kNN Prototyping Schemes for Embedded Human Activity Recognition with Online Learning. Computers 2020, 9, 96. [CrossRef]
  27. Rokach, L.; Maimon, O. Data Mining with Decision Trees: Theory and Applications; Series in Machine Perception and Artificial Intelligence; WSPC: Singapore, 2014.
  28. Xu, S. Bayesian Naïve Bayes classifiers to text classification. J. Inf. Sci. 2018, 44, 48-59. [CrossRef]
  29. Ordóñez, F.J.; Roggen, D. Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 2016, 16, 115. [CrossRef]
  30. Bae, J.; Tomizuka, M. Gait phase analysis based on a Hidden Markov Model. Mechatronics 2011, 21, 961-970. [CrossRef]
  31. Yang, J.B.; Nguyen, M.N.; San, P.P.; Li, X.L.; Krishnaswamy, S. Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina, 25-31 July 2015; pp. 3995-4001.
  32. Ben-Hur, A.; Weston, J. A User's Guide to Support Vector Machines. Methods Mol. Biol. 2009, 609, 223-239. [CrossRef]
  33. Sprager, S.; Juric, M.B. An Efficient HOS-Based Gait Authentication of Accelerometer Data. IEEE Trans. Inf. Forensics Secur. 2015, 10, 1486-1498. [CrossRef]
  34. Vaswani, A.; Shazeer, N.M.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, L.; Polosukhin, I. Attention is All you Need. Conference on Neural Information Processing Systems (NIPS 2017) 2017, CA, USA. arXiv 2016, arXiv:1706.03762.
  35. Tajbakhsh, N.; Shin, J.Y.; Gurudu, S.R.; Hurst, R.T.; Kendall, C.B.; Gotway, M.B.; Liang, J. Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? IEEE Trans. Med. Imaging 2016, 35, 1299-1312. [CrossRef] [PubMed]
  36. Alharthi, A.S.; Yunas, S.U.; Ozanyan, K.B. Deep Learning for Monitoring of Human Gait: A Review. IEEE Sens. J. 2019, 19, 9575-9591. [CrossRef]
  37. Kiranyaz, S.; Avci, O.; Abdeljaber, O.; Ince, T.; Gabbouj, M.; Inman, D.J. 1D convolutional neural networks and applications: A survey. Mech. Syst. Signal Process 2020, 151, 107398. [CrossRef]
  38. Bisong, E. Recurrent Neural Networks (RNNs). In Building Machine Learning and Deep Learning Models on Google Cloud Platform; Apress: Berkeley, CA, USA, 2019; pp. 443-473. [CrossRef]
  39. Lempereur, M.; Rousseau, F.; Rémy-Néris, O.; Pons, C.; Houx, L.; Quellec, G.; Brochard, S. A new deep learning-based method for the detection of gait events in children with gait disorders: Proof-of-concept and concurrent validity. J. Biomech. 2020, 98, 109490.
  40. Hu, Y.; Wong, Y.; Wei, W.; Du, Y.; Kankanhalli, M.; Geng, W. A novel attention-based hybrid CNN-RNN architecture for sEMG-based gesture recognition. PLoS ONE 2018, 13, e0206049. [CrossRef]
  41. Wang, K.; He, J.; Zhang, L. Attention-Based Convolutional Neural Network for Weakly Labeled Human Activities' Recognition with Wearable Sensors. IEEE Sens. J. 2019, 19, 7598-7604. [CrossRef]
  42. Salyers, J.B.; Dong, Y.; Gai, Y. Continuous Wavelet Transform for Decoding Finger Movements from Single-Channel EEG. IEEE Trans. Biomed. Eng. 2019, 66, 1588-1597. [CrossRef]
  43. Ji, N.; Zhou, H.; Guo, K.; Samuel, O.W.; Huang, Z.; Xu, L.; Li, G. Appropriate Mother Wavelets for Continuous Gait Event Detection Based on Time-Frequency Analysis for Hemiplegic and Healthy Individuals. Sensors 2019, 19, 3462. [CrossRef]
  44. Shilane, D. Automated Feature Reduction in Machine Learning. In Proceedings of the IEEE 12th Annual Computing and Communication Workshop and Conference, Virtual, 26-29 January 2022. [CrossRef]
  45. Sun, S.-L.; Deng, Z.-L. Multi-sensor optimal information fusion Kalman filter. Automatica 2004, 40, 1017-1023. [CrossRef]
  46. Narkhede, P.; Poddar, S.; Walambe, R.; Ghinea, G.; Kotecha, K. Cascaded Complementary Filter Architecture for Sensor Fusion in Attitude Estimation. Sensors 2021, 21, 1937. [CrossRef] [PubMed]
  47. Aguiar-Conraria, L.; Soares, M.J. The continuous wavelet transform: Moving beyond uni-and bivariate analysis. J. Econ. Surv. 2014, 28, 344-375. [CrossRef]
  48. Chen, H.; Lu, F.; He, B. Topographic property of backpropagation artificial neural network: From human functional connectivity network to artificial neural network. Neurocomputing 2020, 418, 200-210. [CrossRef]
  49. Chen, T.; Xu, R.; He, Y.; Wang, X. Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN. Expert Syst. Appl. 2017, 72, 221-230. [CrossRef]
  50. Zhao, Y.; Yang, R.; Chevalier, G.; Shah, R.C.; Romijnders, R. Applying deep bidirectional LSTM and mixture density network for basketball trajectory prediction. Optik 2017, 158, 266-272. [CrossRef]
  51. Ogawa, A.; Hori, T. Error detection and accuracy estimation in automatic speech recognition using deep bidirectional recurrent neural networks. Speech Commun. 2017, 89, 70-83. [CrossRef]
  52. Yang, J.; Bai, Y.; Lin, F.; Liu, M.; Hou, Z.; Liu, X. A novel electrocardiogram arrhythmia classification method based on stacked sparse auto-encoders and softmax regression. Int. J. Mach. Learn. Cybern. 2018, 9, 1733-1740. [CrossRef]
  53. Zhang, B.; Zhang, H.; Zhao, G.; Lian, J. Constructing a PM2.5 concentration prediction model by combining auto-encoder with Bi-LSTM neural networks. Environ. Model. Softw. 2020, 124, 104600. [CrossRef]
  54. Zhang, H.; Xiao, Z.; Wang, J.; Li, F.; Szczerbicki, E. A Novel IoT-Perceptive Human Activity Recognition (HAR) Approach Using Multihead Convolutional Attention. IEEE Internet Things J. 2019, 7, 1072-1080. [CrossRef]
  55. Hossin, M.; Sulaimn, N. A Review on Evaluation Metrics for Data Classification Evaluations. Int. J. Data Min. Knowl. Manag. Process 2015, 5, 01-11. [CrossRef]
  56. Urakami, H.; Nikaido, Y.; Kuroda, K.; Ohno, H.; Saura, R.; Okada, Y. Forward gait instability in patients with Parkinson's disease with freezing of gait. Neurosci. Res. 2021, 173, 80-89. [CrossRef] [PubMed]
  57. Disclaimer/Publisher's Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.