Academia.eduAcademia.edu

Figure 4 – uploaded by halit apaydin

See full PDFdownloadDownload figure
Figure 4. General structure of a neural network [29]. An ANN isa distributed knowledge treatment system in which performance essentials are alike to the human brain, and is based on a simulated biological neural network [28]. Each neural network has three layers, namely, input, hidden, and output. The input layer is a layer for providing data provided as inputs to the network. The output layer contains values predicted by the network. The hidden layer is the data analysis location. Usually, the number of selected neurons of the layers is obtained by trial and error. The general architecture of the ANN is displayed in Figure 4, where X (x1, X2, .., Xn) = inputs vector, W = connecting weights to the next layer, bj; = bias, and y; is the ANN final output. The activation function converts input signals into output signals. In Figure 4, N inputs are given from x, to xn to the counterpart weights Wy, to Wey. Initially, the weights are multiplied by their inputs, and then they are summed with the amount of bias to obtain u (Fauation (1)):

Figure 4 General structure of a neural network [29]. An ANN isa distributed knowledge treatment system in which performance essentials are alike to the human brain, and is based on a simulated biological neural network [28]. Each neural network has three layers, namely, input, hidden, and output. The input layer is a layer for providing data provided as inputs to the network. The output layer contains values predicted by the network. The hidden layer is the data analysis location. Usually, the number of selected neurons of the layers is obtained by trial and error. The general architecture of the ANN is displayed in Figure 4, where X (x1, X2, .., Xn) = inputs vector, W = connecting weights to the next layer, bj; = bias, and y; is the ANN final output. The activation function converts input signals into output signals. In Figure 4, N inputs are given from x, to xn to the counterpart weights Wy, to Wey. Initially, the weights are multiplied by their inputs, and then they are summed with the amount of bias to obtain u (Fauation (1)):

Related Figures (14)

Table 1. Statistical characteristics of daily streamflow and rainfall.
Table 1. Statistical characteristics of daily streamflow and rainfall.
Figure 1. Location of the study area.
Figure 1. Location of the study area.
For modeling, ANN methods and four different RNN deep-network methods were employed. In total, 70% and 30% of the data were used in training and testing stages, respectively, as indicated in Figure 2. Besides this, several delayed daily mean streamflow are used as input data. For this purpose, the correlation between the time series of streamflows with its delays is obtained, and seven delays (SF t-1, SFt.2, SF 1-3, SFt-4, SFt-5, SF}-6, and SF;-7) are selected as inputs due to high correlation (Figure 3). rhe correlation coefficients are found to vary between 0.63 and 0.85 in seven-day lag conditions. Similar studies [7,8] have also used seven delays in flow time series as inputs.
For modeling, ANN methods and four different RNN deep-network methods were employed. In total, 70% and 30% of the data were used in training and testing stages, respectively, as indicated in Figure 2. Besides this, several delayed daily mean streamflow are used as input data. For this purpose, the correlation between the time series of streamflows with its delays is obtained, and seven delays (SF t-1, SFt.2, SF 1-3, SFt-4, SFt-5, SF}-6, and SF;-7) are selected as inputs due to high correlation (Figure 3). rhe correlation coefficients are found to vary between 0.63 and 0.85 in seven-day lag conditions. Similar studies [7,8] have also used seven delays in flow time series as inputs.
Figure 3. Correlation plot between streamflow and its delays.
Figure 3. Correlation plot between streamflow and its delays.
Figure 5. The flow chart of the modelling processes. *SF: Streamflow, t: delayed day, ANN: Artificial neural network, RNN: Recurrent neural network, Bi-LSTM: Bidirectional long short-term memory, GRU: Gated recurrent unit, LSTM: Long short-term memory, simple RNN: simple recurrent neural networks. Figure 5. The flow chart of the modelling processes. *SF: Streamflow, t: delayed day, ANN: Artificial
Figure 5. The flow chart of the modelling processes. *SF: Streamflow, t: delayed day, ANN: Artificial neural network, RNN: Recurrent neural network, Bi-LSTM: Bidirectional long short-term memory, GRU: Gated recurrent unit, LSTM: Long short-term memory, simple RNN: simple recurrent neural networks. Figure 5. The flow chart of the modelling processes. *SF: Streamflow, t: delayed day, ANN: Artificial
Figure 6. The general structure of recurrent neural network models. (a) Simple RNN, (b) LSTM, (c) GRU, and (d) Bi-LSTM models [19].
Figure 6. The general structure of recurrent neural network models. (a) Simple RNN, (b) LSTM, (c) GRU, and (d) Bi-LSTM models [19].
Figure 7. Structure of networks (a) before and (b) after applying dropout [38].
Figure 7. Structure of networks (a) before and (b) after applying dropout [38].
The learning rate is one of the hyperparameters necessary to find the optimal value and usually takes values between 1 and 1 x 107’. In fact, the learning rate expresses the size of the move steps by the network. Figure 8 compares the changes in the loss function versus the epoch based on the learning rate. Working with a low learning rate, it takes a long time to find the best solution. Besides, if the earning rate is too high, it rejects the optimal mode. Since a high learning rate is advantageous in early iterations, and a low one is advantageous in later iterations, a learning rate that slows down as the algorithm progresses is preferred [39]. Figure 8. Changes in the loss function vs. the epoch by the learning rate [40].
The learning rate is one of the hyperparameters necessary to find the optimal value and usually takes values between 1 and 1 x 107’. In fact, the learning rate expresses the size of the move steps by the network. Figure 8 compares the changes in the loss function versus the epoch based on the learning rate. Working with a low learning rate, it takes a long time to find the best solution. Besides, if the earning rate is too high, it rejects the optimal mode. Since a high learning rate is advantageous in early iterations, and a low one is advantageous in later iterations, a learning rate that slows down as the algorithm progresses is preferred [39]. Figure 8. Changes in the loss function vs. the epoch by the learning rate [40].
where X; is the observation parameter with a mean denoted by X; Y; is the prediction parameter with a mean denoted by Y; N is a number of instances. The more the two first criteria are closer to 1 and the next three values are closer to 0 show the better performance of the model. According to Chiew et al. [41], if NS > 0.90, the simulation is very acceptable; if 0.60 < NS < 0.90, the simulation is acceptable; and if NS < 0.60 as in this case, simulation is unacceptable.
where X; is the observation parameter with a mean denoted by X; Y; is the prediction parameter with a mean denoted by Y; N is a number of instances. The more the two first criteria are closer to 1 and the next three values are closer to 0 show the better performance of the model. According to Chiew et al. [41], if NS > 0.90, the simulation is very acceptable; if 0.60 < NS < 0.90, the simulation is acceptable; and if NS < 0.60 as in this case, simulation is unacceptable.
Table 2. Results of different models *. Different models are run with different iterations. As seen in Table 2, the number of epochs as one of the influential parameters plays a basic role; so in fewer iterations, the accuracy of the model is low, and the error is greater. By increasing the number of iterations, the model gradually converges so that there is not a significant difference between the results of 300 and 500 epochs. For this reason, it is decided that a maximum of 500 epochs is sufficient. Another parameter that influences the accuracy of the models is the number of neurons in the hidden layers. If it is low, the model will not be able to simulate correctly, and if it is too high, there is a risk of overfitting. This problem is solved by using the dropout method, which in fact deactivates several neurons. The values of the learning rate (LR) and decay are also presented in Table 2. Decay occurs due to how much the learning rate is reduced in each step. As seen in Table 2, five different evaluation criteria are used to compare five different prediction methods. Each method has the best result with 300 or 500 epochs. The performance of the testing stage is generally 1-5% lower than that of the training stage. According to Table 2, among the methods used, the LSTM method performs best in 500 epochs with an accuracy of CC = 87% in the testing stage. This network has long-term memory, and its forget gate specifies how much previous memory is kept. The first step is the multiplication of input data in weights and then its summation with the bias, followed by output. In this step, the output is likely to be very different from the actual output. Therefore, errors are returned backward to update the weights and also biases. Comparison of some metrics by all methods used in the study are given in Table 3. In this table,
Table 2. Results of different models *. Different models are run with different iterations. As seen in Table 2, the number of epochs as one of the influential parameters plays a basic role; so in fewer iterations, the accuracy of the model is low, and the error is greater. By increasing the number of iterations, the model gradually converges so that there is not a significant difference between the results of 300 and 500 epochs. For this reason, it is decided that a maximum of 500 epochs is sufficient. Another parameter that influences the accuracy of the models is the number of neurons in the hidden layers. If it is low, the model will not be able to simulate correctly, and if it is too high, there is a risk of overfitting. This problem is solved by using the dropout method, which in fact deactivates several neurons. The values of the learning rate (LR) and decay are also presented in Table 2. Decay occurs due to how much the learning rate is reduced in each step. As seen in Table 2, five different evaluation criteria are used to compare five different prediction methods. Each method has the best result with 300 or 500 epochs. The performance of the testing stage is generally 1-5% lower than that of the training stage. According to Table 2, among the methods used, the LSTM method performs best in 500 epochs with an accuracy of CC = 87% in the testing stage. This network has long-term memory, and its forget gate specifies how much previous memory is kept. The first step is the multiplication of input data in weights and then its summation with the bias, followed by output. In this step, the output is likely to be very different from the actual output. Therefore, errors are returned backward to update the weights and also biases. Comparison of some metrics by all methods used in the study are given in Table 3. In this table,
The loss function, observational streamflow values and predicted values for training and testing ges computed by various methods are displayed in Figure 9. When the loss function and time-series iphs are examined, the most successful methods are LSTM and GRU. The loss charts of testing and ining stages of both methods overlap at the highest epoch value. According to the loss function graphs sure 9), with the reduction of the modeling error for the training stage, the error of the testing stage reases, and the distance between the two graph lines reduces. Therefore, it can be said that the dropou ction helps to prevent network overfitting. Besides, considering the changes in the loss function value sus epochs (and according to Figure 8) can ensure that the learning rate is selected correctly.
The loss function, observational streamflow values and predicted values for training and testing ges computed by various methods are displayed in Figure 9. When the loss function and time-series iphs are examined, the most successful methods are LSTM and GRU. The loss charts of testing and ining stages of both methods overlap at the highest epoch value. According to the loss function graphs sure 9), with the reduction of the modeling error for the training stage, the error of the testing stage reases, and the distance between the two graph lines reduces. Therefore, it can be said that the dropou ction helps to prevent network overfitting. Besides, considering the changes in the loss function value sus epochs (and according to Figure 8) can ensure that the learning rate is selected correctly.
Table 3. Comparison of estimated values with observed values for all methods. Figure 10 shows the scatter of observed values versus the predicted values for both training and testing stages. Regarding the streamflow graphs (Figures 9 and 10), the networks yield poor results in peak periods and cannot simulate these periods well, except LSTM. One major reason of this is that streamflow abundance has very few high values, and the network cannot properly learn them. This happens even though the streamflow abundance has several low values, so the model can correctly and accurately be learned in the training stage. According to the data, the difference between the minimum and the maximum flow values is high, and the validity of these values is investigated during the study period based on precipitation in the region. In periods where peak streamflow is observed, the maximum rainfall is observed with a significant amount that indicates the correctness of peak flow values. However, among the methods applied, the LSTM network is better than other methods, and is able to simulate peak flow periods fairly. As noted above, the greatest feature of this network is its skill to learn long-term dependencies, and the forget gate makes the network keep or forget the desired amount of previous memory and thus helps to improve the modeling. The GRU method is similar to the LSTM method, and its results are close to those of LSTM. In contrast, the other methods failed to simulate peak flow values. The scatter plots in Figure 10 indicate a relatively small dispersion of observed and predicted data in training and testing stages compared to other methods. Kratzert et al. [19] used deep LSTM networks in similar research and demonstrated that these networks perform better than other networks and can be used in hydrological simulations as an application method.
Table 3. Comparison of estimated values with observed values for all methods. Figure 10 shows the scatter of observed values versus the predicted values for both training and testing stages. Regarding the streamflow graphs (Figures 9 and 10), the networks yield poor results in peak periods and cannot simulate these periods well, except LSTM. One major reason of this is that streamflow abundance has very few high values, and the network cannot properly learn them. This happens even though the streamflow abundance has several low values, so the model can correctly and accurately be learned in the training stage. According to the data, the difference between the minimum and the maximum flow values is high, and the validity of these values is investigated during the study period based on precipitation in the region. In periods where peak streamflow is observed, the maximum rainfall is observed with a significant amount that indicates the correctness of peak flow values. However, among the methods applied, the LSTM network is better than other methods, and is able to simulate peak flow periods fairly. As noted above, the greatest feature of this network is its skill to learn long-term dependencies, and the forget gate makes the network keep or forget the desired amount of previous memory and thus helps to improve the modeling. The GRU method is similar to the LSTM method, and its results are close to those of LSTM. In contrast, the other methods failed to simulate peak flow values. The scatter plots in Figure 10 indicate a relatively small dispersion of observed and predicted data in training and testing stages compared to other methods. Kratzert et al. [19] used deep LSTM networks in similar research and demonstrated that these networks perform better than other networks and can be used in hydrological simulations as an application method.
Figure 9. Loss function plots (a—e), observed and computed time series (f-j) for different methods in training and testing stages.
Figure 9. Loss function plots (a—e), observed and computed time series (f-j) for different methods in training and testing stages.
Related topics:
Computer ScienceTime Series Analysis and ForecastingLong Short Term MemoryLong short term memory cells
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved