Academia.eduAcademia.edu

Figure 8 – uploaded by Riko Safaric

See full PDFdownloadDownload figure
Table 7. Sensor importance evaluation. Table 7 presents the results from evaluating the CNN + RNN algorithm after training with different combinations of Complementary Filter (CF), Accelerometer (ACC), Gyro- scope (GYRO) and Strain Gauge (SG) RAW sensor information.

Table 7 Sensor importance evaluation. Table 7 presents the results from evaluating the CNN + RNN algorithm after training with different combinations of Complementary Filter (CF), Accelerometer (ACC), Gyro- scope (GYRO) and Strain Gauge (SG) RAW sensor information.

Related Figures (21)

running for robust gait monitoring, analysis and recognition. A wearable system with sensors, a processing unit and communication, packed in a small lightweight housing is usually desired for long-term daily use. The important criteria for a wearable system are also low power consumption, sufficient external memory storage, and a user-friendly human interface for on-line data visualization and monitoring. In the case of Machine Learning (ML) methods, many of them now enable feature extraction. A combination of different learning methods can be used to recognize and extract human walking features adequately. A tradeoff between code complexity and real-time operation needs to be considered as far as implementation is concerned. The human gait is a cyclical process, with the joint operation of bones, muscles and the nervous system trying to maintain static and dynamic balance of an upright body position while in motion. The cycle can be defined as the time between two consecutive contacts of the foot striking the ground while walking and has two states, namely stance when the foot is firmly on the ground, and swing when the foot is lifted from the ground, as seen in Figure 1. Furthermore, gait phases can be divided into durations of certain events (left and right oot up/down, limb stance/swing, and limb turn) [5], and classified according to the area of nterest (sport, health, prosthetics, and daily life) [6-8], type of sensors used (floor sensors, risual, inertial or other sensors) [9,10], and last but not least, its placement (shoe, ankle, imb, and hip/back) [11,12]. The gait analysis in sports can improve walking/running echniques, prevent long-term hip, knee and ankle injuries, improve sport results, and iid the development of custom-made athlete’s textiles or sport shoes [6]. In medical pplications, a gait analysis can result in successful recognition of various gait disorders, like ‘reezing Of Gait (FOG) in Parkinson’s Disease (PD) or Cerebral Palsy (CP) patients [3]. In rosthetics applications, the gait analysis can help disabled patients after limb amputation o improve feedback in walking with a robotic prosthesis [8,13].
running for robust gait monitoring, analysis and recognition. A wearable system with sensors, a processing unit and communication, packed in a small lightweight housing is usually desired for long-term daily use. The important criteria for a wearable system are also low power consumption, sufficient external memory storage, and a user-friendly human interface for on-line data visualization and monitoring. In the case of Machine Learning (ML) methods, many of them now enable feature extraction. A combination of different learning methods can be used to recognize and extract human walking features adequately. A tradeoff between code complexity and real-time operation needs to be considered as far as implementation is concerned. The human gait is a cyclical process, with the joint operation of bones, muscles and the nervous system trying to maintain static and dynamic balance of an upright body position while in motion. The cycle can be defined as the time between two consecutive contacts of the foot striking the ground while walking and has two states, namely stance when the foot is firmly on the ground, and swing when the foot is lifted from the ground, as seen in Figure 1. Furthermore, gait phases can be divided into durations of certain events (left and right oot up/down, limb stance/swing, and limb turn) [5], and classified according to the area of nterest (sport, health, prosthetics, and daily life) [6-8], type of sensors used (floor sensors, risual, inertial or other sensors) [9,10], and last but not least, its placement (shoe, ankle, imb, and hip/back) [11,12]. The gait analysis in sports can improve walking/running echniques, prevent long-term hip, knee and ankle injuries, improve sport results, and iid the development of custom-made athlete’s textiles or sport shoes [6]. In medical pplications, a gait analysis can result in successful recognition of various gait disorders, like ‘reezing Of Gait (FOG) in Parkinson’s Disease (PD) or Cerebral Palsy (CP) patients [3]. In rosthetics applications, the gait analysis can help disabled patients after limb amputation o improve feedback in walking with a robotic prosthesis [8,13].
Figure 2. Sketch of the proposed system. The motivation of the paper is to develop a robust and lightweight system capable of human gait activity monitoring, analysis and recognition, with the aim to improve a subject’s QOL if applied in the fields of medicine, sports, or prosthetics. The proposed system (Figure 2) consists of the following components: A gait motion data acquisition system, time-series data collection, data pre-processing and data processing (classification).
Figure 2. Sketch of the proposed system. The motivation of the paper is to develop a robust and lightweight system capable of human gait activity monitoring, analysis and recognition, with the aim to improve a subject’s QOL if applied in the fields of medicine, sports, or prosthetics. The proposed system (Figure 2) consists of the following components: A gait motion data acquisition system, time-series data collection, data pre-processing and data processing (classification).
Figure 3. (a) Wearing the proposed gait motion data acquisition system; (b) close-up picture of the elastic strip equipped with sensors; (c) bare sensor unit without the elastic strip. The sensors are covered with elastic glue to increase longevity. Figure 3. (a) Wearing the proposed gait motion data acquisition system; (b) close-up picture of the
Figure 3. (a) Wearing the proposed gait motion data acquisition system; (b) close-up picture of the elastic strip equipped with sensors; (c) bare sensor unit without the elastic strip. The sensors are covered with elastic glue to increase longevity. Figure 3. (a) Wearing the proposed gait motion data acquisition system; (b) close-up picture of the
The stream of sensor data is stored into a matrix with a rate of 40 Hz (every 25 ms), where each new measurement is stored into a new matrix row. The recorded gait data matrix is visualized (Figure 5), and different gait activities can be observed throughout time. Figure 5. Recorded time-series gait data visualized (the displayed sensor information has already
The stream of sensor data is stored into a matrix with a rate of 40 Hz (every 25 ms), where each new measurement is stored into a new matrix row. The recorded gait data matrix is visualized (Figure 5), and different gait activities can be observed throughout time. Figure 5. Recorded time-series gait data visualized (the displayed sensor information has already
Table 1. Features and their characteristics.
Table 1. Features and their characteristics.
Figure 6. CWT time to frequency domain transformation visualized. (a) Presents the normalized X axis of the complementary filter; (b) visualizes the result from the CWT algorithm; (c) presents the stacked CWT result of each sensor signal. Human gait activity generally consists of frequencies between 0.5-10 Hz, therefore a frequency window of 0.2-20 Hz was selected to incorporate all the necessary information (Figure 6). CWT is a continuous 1-D transform function, so the input must be a 1-D single variable vector, from which 27 new frequency features are generated, where each feature corresponds to a specific frequency, with its value indicating its amplitude. A frequency window of 0.2-20 Hz gets divided into 27 smaller ‘frequency windows-features’. CWT was performed for all 20 acquired sensor signals independently, and later combined, generating a matrix containing 540 variables that we used to train the ML algorithms.
Figure 6. CWT time to frequency domain transformation visualized. (a) Presents the normalized X axis of the complementary filter; (b) visualizes the result from the CWT algorithm; (c) presents the stacked CWT result of each sensor signal. Human gait activity generally consists of frequencies between 0.5-10 Hz, therefore a frequency window of 0.2-20 Hz was selected to incorporate all the necessary information (Figure 6). CWT is a continuous 1-D transform function, so the input must be a 1-D single variable vector, from which 27 new frequency features are generated, where each feature corresponds to a specific frequency, with its value indicating its amplitude. A frequency window of 0.2-20 Hz gets divided into 27 smaller ‘frequency windows-features’. CWT was performed for all 20 acquired sensor signals independently, and later combined, generating a matrix containing 540 variables that we used to train the ML algorithms.
Figure 7. Auto encoder structure. the ability to learn efficient data compression and decompression. The weights and biases of the network’s neurons are adjusted to achieve minimal reconstruction loss (comparing input features and reconstructed features). The AE consists of a minimum of 3 layers: An encoder, a bottleneck and a decoder (Figure 7). The bottleneck layer includes a smaller number of neurons (the bottleneck itself), which need to learn the most vital information encompassed in the input features to reconstruct them successfully through the decoding process. Once the AE learns feature reconstruction, only the trained encoder and bottleneck layer are needed for data compression (feature reduction). The data label information was used as a ‘target’ for training the ML algorithm (a target indicating the desired output of the algorithm), so that when the trained ML algorithm is tested on an ‘unseen’ gait recording, the algorithm outputs (reveals) different gait activities within it (essentially a trained ML algorithm then creates data labels).
Figure 7. Auto encoder structure. the ability to learn efficient data compression and decompression. The weights and biases of the network’s neurons are adjusted to achieve minimal reconstruction loss (comparing input features and reconstructed features). The AE consists of a minimum of 3 layers: An encoder, a bottleneck and a decoder (Figure 7). The bottleneck layer includes a smaller number of neurons (the bottleneck itself), which need to learn the most vital information encompassed in the input features to reconstruct them successfully through the decoding process. Once the AE learns feature reconstruction, only the trained encoder and bottleneck layer are needed for data compression (feature reduction). The data label information was used as a ‘target’ for training the ML algorithm (a target indicating the desired output of the algorithm), so that when the trained ML algorithm is tested on an ‘unseen’ gait recording, the algorithm outputs (reveals) different gait activities within it (essentially a trained ML algorithm then creates data labels).
Figure 8. Visualization of the CNN + RNN algorithm architecture. A simple CNN + RNN algorithm combines the advantages of both architectures. The Convolutional layers have learnable filter parameters which can learn to produce the most meaningful feature maps for successful classification by later recurrent layers of the algorithm [35-39]. The 1D convolutional layer allows for processing of time-series data, whereas the 2D and 3D convolutional layers are designed for image processing. The CNN + RNWN algorithm architecture is presented in Figure 8. In Deep Learning, the attention mechanism learns to model the importance inside the learning data through optimizing attention weights ‘a,,’. The attention mechanism learns to enhance the most important aspects of the data, while diminishing other, less important
Figure 8. Visualization of the CNN + RNN algorithm architecture. A simple CNN + RNN algorithm combines the advantages of both architectures. The Convolutional layers have learnable filter parameters which can learn to produce the most meaningful feature maps for successful classification by later recurrent layers of the algorithm [35-39]. The 1D convolutional layer allows for processing of time-series data, whereas the 2D and 3D convolutional layers are designed for image processing. The CNN + RNWN algorithm architecture is presented in Figure 8. In Deep Learning, the attention mechanism learns to model the importance inside the learning data through optimizing attention weights ‘a,,’. The attention mechanism learns to enhance the most important aspects of the data, while diminishing other, less important
Figure 9. Visualization of the CNNA + RNN algorithm architecture.
Figure 9. Visualization of the CNNA + RNN algorithm architecture.
Table 2. Dataset activity distribution. Relatively high activity imbalances in datasets can be observed in Table 2. Inspecting the mean values of activity distributions, running was the sparsest activity (at only 6.8%), while 58.8% of the datasets consisted of walking.
Table 2. Dataset activity distribution. Relatively high activity imbalances in datasets can be observed in Table 2. Inspecting the mean values of activity distributions, running was the sparsest activity (at only 6.8%), while 58.8% of the datasets consisted of walking.
Table 3. ML algorithm parameters.
Table 3. ML algorithm parameters.
Table 3. Cont.
Table 3. Cont.
Table 4. Averaged cross-validated evaluation metrics for the corresponding nine classifiers Table 5. Standard deviation of cross-validated results. od In the remainder of this subsection, the results of the experiments are explained and presented in detail (Tables 4—7), while their further discussion is summarized in the next section. The ML performance of nine algorithms compared regarding the five evaluation metrics is presented in Table 4. The results are calculated for the CWT and RAW datasets separately. Furthermore, it is desired to use the RAW dataset for classification, since we do not have to calculate CWT at all. The best results according to each classification measure are presented as bold in the tables.
Table 4. Averaged cross-validated evaluation metrics for the corresponding nine classifiers Table 5. Standard deviation of cross-validated results. od In the remainder of this subsection, the results of the experiments are explained and presented in detail (Tables 4—7), while their further discussion is summarized in the next section. The ML performance of nine algorithms compared regarding the five evaluation metrics is presented in Table 4. The results are calculated for the CWT and RAW datasets separately. Furthermore, it is desired to use the RAW dataset for classification, since we do not have to calculate CWT at all. The best results according to each classification measure are presented as bold in the tables.
Table 6. Execution time of the classification algorithms.
Table 6. Execution time of the classification algorithms.
Table 8. RAW dataset evaluation.
Table 8. RAW dataset evaluation.
Figure 10. Visualization of the CNNA + RNN algorithm’s output on a test RAW 250 s gait record- ing; (a) Complementary filter information from ‘unseen’ gait dataset visualized; (b) illustrates the algorithm’s output on the ‘unseen’ dataset; (c) presents a comparison of the data label and algorithm’s output. Table 9. CWT dataset evaluation.
Figure 10. Visualization of the CNNA + RNN algorithm’s output on a test RAW 250 s gait record- ing; (a) Complementary filter information from ‘unseen’ gait dataset visualized; (b) illustrates the algorithm’s output on the ‘unseen’ dataset; (c) presents a comparison of the data label and algorithm’s output. Table 9. CWT dataset evaluation.
Figure 11. Visualization of the CNNA + RNN algorithm’s output on the test CWT dataset of the PD patient, that experienced five FOG episodes; (a) Complementary filter information visualized from an ‘unseen’ gait dataset; (b) illustrates the algorithm’s output on the ‘unseen’ dataset; (c) presents a comparison of the data label and the algorithm’s output. Figure 11. Visualization of the CNNA + RNN algorithm’s output on the test CWT dataset of the PD The PD patient's 24 min gait recording was split into a train dataset (60%) and test dataset (40%). The results of the latter are visualized in Figure 11a, alongside the algorithm’s output in Figure 11b. Figure 11c compares the algorithm’s output with the data label.
Figure 11. Visualization of the CNNA + RNN algorithm’s output on the test CWT dataset of the PD patient, that experienced five FOG episodes; (a) Complementary filter information visualized from an ‘unseen’ gait dataset; (b) illustrates the algorithm’s output on the ‘unseen’ dataset; (c) presents a comparison of the data label and the algorithm’s output. Figure 11. Visualization of the CNNA + RNN algorithm’s output on the test CWT dataset of the PD The PD patient's 24 min gait recording was split into a train dataset (60%) and test dataset (40%). The results of the latter are visualized in Figure 11a, alongside the algorithm’s output in Figure 11b. Figure 11c compares the algorithm’s output with the data label.
Table 10. Evaluation metrics for four classifiers.
Table 10. Evaluation metrics for four classifiers.
Table 11. Sensor importance evaluation for FOG detection. As is evident in Table 11, the CNN + RNN algorithm trained with the combined CWT data, obtained by all four sensors, showed superiority, although it also performed relatively well for some of the other tested sensor combinations, e.g., training the ML algorithm using ACC, GYRO and ACC, CF combinations produced the highest precision and sensitivity scores, respectively.
Table 11. Sensor importance evaluation for FOG detection. As is evident in Table 11, the CNN + RNN algorithm trained with the combined CWT data, obtained by all four sensors, showed superiority, although it also performed relatively well for some of the other tested sensor combinations, e.g., training the ML algorithm using ACC, GYRO and ACC, CF combinations produced the highest precision and sensitivity scores, respectively.
Related topics:
Computer ScienceArtificial IntelligenceAnalytical ChemistryMachine LearningGait AnalysisElectrical and Electronic Engineering
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved