Academia.eduAcademia.edu

Figure 2 – uploaded by International Journal of Electrical and Computer Engineering (IJECE)

See full PDFdownloadDownload figure
Int J Elec & Comp Eng, Vol. 13, No. 2, April 2023: 2023-2029

Table 2 Int J Elec & Comp Eng, Vol. 13, No. 2, April 2023: 2023-2029

Related Figures (3)

There are nine features available for each instance of data which include triaxial acceleration from the accelerometer (total acceleration), the estimated body acceleration and the triaxial angular velocity from
There are nine features available for each instance of data which include triaxial acceleration from the accelerometer (total acceleration), the estimated body acceleration and the triaxial angular velocity from
3.3. Multi-head self-attention architecture 3.3.1. Multi-head self-attention layer The LSTM baseline model has 3 layers of LSTM. The first LSTM layer has 32 hidden units. The second LSTM layer has 64 hidden units. The third LSTM layer has 32 hidden units. The final time step of the third LSTM layer is connected to a fully connected layer. The fully connected layer has 6 units corresponding to 6 classes of activities and uses a SoftMax activation function. Figure 2 summarizes the LSTM model we used.
3.3. Multi-head self-attention architecture 3.3.1. Multi-head self-attention layer The LSTM baseline model has 3 layers of LSTM. The first LSTM layer has 32 hidden units. The second LSTM layer has 64 hidden units. The third LSTM layer has 32 hidden units. The final time step of the third LSTM layer is connected to a fully connected layer. The fully connected layer has 6 units corresponding to 6 classes of activities and uses a SoftMax activation function. Figure 2 summarizes the LSTM model we used.
Figure 3. Multi-head self-attention model used for time series activity recognition Figure 3 shows the multi-head self-attention model for human activity recognition for one daté instance. A data instance of 128x9 is expanded in dimension to 128x128 after passing through a fully connected layer (labelled FC in Figure 3) with ReLU activation. Then, the matrix of 128x128 is passed tc 2 blocks of “4-head self-attention block”. Each 4-head self-attention block consists of 2 layers. The first laye1 is a 4-head self-attention layer described in Part 1 of this subsection. Each individual self-attention layet outputs a 128x128 matrix. The 4-head self-attention layer results in four 128x128 matrices which is ther concatenated to a matrix of 128x512. The output matrix of this concatenated matrix is passed to a fully connected layer with 128 hidden units, which then outputs a 128x128 matrix. The output matrix of 2 blocks of self-attention is matrix of 128x128, which is then unrolled to a column vector of 16384 dimensions. The column vector is then passed through a FC layer with 128 units using ReLU and then to the prediction laye1 with six units with SoftMax activation. The output is a vector of 6. Finally, we added dropout for some FC layer. Table 2 summarizes the model and listed the number of parameters for each layer.
Figure 3. Multi-head self-attention model used for time series activity recognition Figure 3 shows the multi-head self-attention model for human activity recognition for one daté instance. A data instance of 128x9 is expanded in dimension to 128x128 after passing through a fully connected layer (labelled FC in Figure 3) with ReLU activation. Then, the matrix of 128x128 is passed tc 2 blocks of “4-head self-attention block”. Each 4-head self-attention block consists of 2 layers. The first laye1 is a 4-head self-attention layer described in Part 1 of this subsection. Each individual self-attention layet outputs a 128x128 matrix. The 4-head self-attention layer results in four 128x128 matrices which is ther concatenated to a matrix of 128x512. The output matrix of this concatenated matrix is passed to a fully connected layer with 128 hidden units, which then outputs a 128x128 matrix. The output matrix of 2 blocks of self-attention is matrix of 128x128, which is then unrolled to a column vector of 16384 dimensions. The column vector is then passed through a FC layer with 128 units using ReLU and then to the prediction laye1 with six units with SoftMax activation. The output is a vector of 6. Finally, we added dropout for some FC layer. Table 2 summarizes the model and listed the number of parameters for each layer.
Related topics:
Human Activity RecognitionSelf-AttentionLong Short Term Memory
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved