deep learnig

This paper provides a comprehensive overview of MoE, covering its fundamental principles, architectural variations, advantages, limitations, and potential future directions. We delve into the core concepts of MoE, including the gating network, expert networks, and routing mechanisms, and discuss how these components work together to achieve specialization and efficiency. We also examine the application of MoE in models like GPT-4 and Mixtral, highlighting their impact on the field of AI. We cover theoretical foundations, hardware and software innovations, real-world deployments, and the evolving landscape of MoE research. This paper furthur provides a comprehensive survey of MoE architectures, tracing their evolution from early neural network implementations to modern large-scale applications in language models, time series forecasting, and tabular data analysis. Next, the paper examines how machine learning is applied to natural language processing, computer vision, finance and healthcare. We examine major problems, including routing imbalance, memory fragmentation and instability during training, by checking newly proposed answers found in research papers. After all, we summarize possible future areas of study and discuss how MoE models could transform the world of artificial intelligence moving forward. All the results in this paper are taken from the cited work.

In the rapidly evolving landscape of artificial intelligence, code generation has emerged as a critical frontier for automating software development, optimizing workflows, and democratizing programming expertise. Among the pioneers in this domain, DeepSeek-a series of large language models (LLMs) with a focus on Mixture-of-Experts (MoE) architecture-has gained global attention for its unparalleled speed, precision, and adaptability. This survey dissects the technological innovations behind DeepSeek's success, exploring how its modular architecture and specialized "experts" enable lightning-fast code generation while maintaining state-of-the-art performance. We contextualize DeepSeek's contributions within the broader AI ecosystem, analyze its benchmarks against competitors, and discuss its implications for the future of software engineering and AI-driven creativity.

strive to develop a reliable and accurate forecasting model for streamflow. In this study, the novel combination of the adaptive neuro-fuzzy inference system (ANFIS) model with the shuffled frog leaping algorithm (SFLA) is proposed. Historical streamflow data of two different rivers were collected to examine the performance of the proposed model. To evaluate the performance of the proposed ANFIS-SFLA model, six different scenarios for the model input-output architecture were investigated. The results show that the proposed ANFIS-SFLA model (r = 0.88; NS = 0.88; RMSE = 141.39; MAE = 88.94; MAPE = 35.19%) significantly improved the forecasting accuracy and outperformed the classic ANFIS model (r = 0.83; NS = 0.83; RMSE = 167.81; MAE = 115.83; MAPE = 42.59%). The proposed model could be generalized and applied in different rivers worldwide.

Face recognition is of pronounced significance to real-world applications such as video surveillance systems, human computing interaction, and security systems. This biometric authenticating system encompasses rich real human face characteristics. As such, it has been one of the important research topics in computer vision. Face recognition systems based on deep learning approaches suffer from internal covariate shift problems that cause gradients to explode or gradient disappearance, which leads to improper network training. Improper network training causes network overfitting and computational load. This reduces recognition accuracy and slows down network speed. This paper proposes a modified preactivation batch normalization convolutional neural network by adding a batch normalization layer after each convolutional layer within each of the four convolutional units of the proposed model. The performance of the proposed model is validated with a new dataset, AS-Darmaset, which is b...

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

In this paper, a TSK-type Neuro-fuzzy Inference System that combines the features of fuzzy sets and neural networks has been applied for the identification of MIMO systems. The procedure of adapting parameters in TSK model employs a Shuffled Frog Leaping Algorithm (SFLA) which is inspired from the memetic evolution of a group of frogs when seeking for food. To demonstrate the accuracy and effectiveness of the proposed controller, two nonlinear systems have been considered as the MIMO plant, and results have been compared with other learning methods based on Particle Swarm Optimization algorithm (PSO) and Genetic Algorithm (GA).

This paper examines the methods of increasing software efficiency based on soft computing technology by analyzing the benefits and challenges of the components used in software development. The functions and features of machine learning, comprising neural networks, perceptron, support vector machines, and fuzzy logic are highlighted and explained, along with those of evolutionary computation (e.g., evolutionary and genetic algorithms) and probability. Recommendations are presented in conclusion. Contribution/Originality: This paper is one of only a few to investigate and analyze the problems of soft computing technologies. By reviewing the utilization of new estimation methodologies, it contributes to the existing literature and comparative analyses.

Feature selection and attribute reduction are crucial problems, and widely used techniques in the field of machine learning, data mining and pattern recognition to overcome the well-known phenomenon of the Curse of Dimensionality, by either selecting a subset of features or removing unrelated ones. This paper presents a new feature selection method that efficiently carries out attribute reduction, thereby selecting the most informative features of a dataset. It consists of two components: 1) a measure for feature subset evaluation, and 2) a search strategy. For the evaluation measure, we have employed the fuzzy-rough dependency degree (FRFDD) in the lower approximation-based fuzzy-rough feature selection (L-FRFS) due to its effectiveness in feature selection. As for the search strategy, a new version of a binary shuffled frog leaping algorithm is proposed (B-SFLA). The new feature selection method is obtained by hybridizing the B-SFLA with the FRDD. Non-parametric statistical tests are conducted to compare the proposed approach with several existing methods over twenty two datasets, including nine high dimensional and large ones, from the UCI repository. The experimental results demonstrate that the B-SFLA approach significantly outperforms other metaheuristic methods in terms of the number of selected features and the classification accuracy.

Face recognition is of pronounced significance to real-world applications such as video surveillance systems, human computing interaction, and security systems. This biometric authenticating system encompasses rich real human face characteristics. As such, it has been one of the important research topics in computer vision. Face recognition systems based on deep learning approaches suffer from internal covariate shift problems that cause gradients to explode or gradient disappearance, which leads to improper network training. Improper network training causes network overfitting and computational load. This reduces recognition accuracy and slows down network speed. This paper proposes a modified preactivation batch normalization convolutional neural network by adding a batch normalization layer after each convolutional layer within each of the four convolutional units of the proposed model. The performance of the proposed model is validated with a new dataset, AS-Darmaset, which is b...

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

The inspiration behind the huge attention given to face recognition systems by the research community and computer vision specialists is the need to enhance face recognition systems' effectiveness, accuracy rate, and speed. The complexity of recognizing the human face by machines due to different variations in poses, illumination, age, facial expression, occlusion, personal appearance, and different cosmetic effects makes face recognition more challenging. However, this makes it difficult to implement a robust computational system. The study's main goal is to enhance the current deep learning approaches for face recognition applications using an enhanced and efficient hybrid deep learning method that involves multi-layer CNN and SVM. The model is encompassed with a newly developed middle block convolutional regularization algorithm (MBCRA) and a preactivation batch normalization method for computational stability and convergence speed. The combination of both CNN and SVM enables the system to obtain more significant face features from the images of the proposed AS_Darmaset. The database has six classes of images. Each class contains face images with specific variation problems. The experimental results demonstrate that the multi-layer CNN+SVM has a 99.87% accuracy, and the comparative analysis shows that the proposed model is more resilient for face image classification under unconstrained settings than the most developed deep learning model for face recognition.

Energy performance analysis in buildings is becoming more and more highlighted, due to the increasing trend of energy consumption in the building sector. Many studies have declared the great potential of soft computing for this analysis. A particular methodology in this sense is employing hybrid machine learning that copes with the drawbacks of single methods. In this work, an optimized version of a popular machine learning model, namely feed-forward neural network (FFNN) is used for simultaneously predicting annual thermal energy demand (ATED) and annual weighted average discomfort degree-hours (WADDH) by analyzing eleven input factors that represent the building circumstances. The optimization task is carried out by a multi-tracker optimization algorithm (MTOA) which is a powerful metaheuristic algorithm. Moreover, three benchmark algorithms including the slime mould algorithm (SMA), seeker optimization algorithm (SOA), and vortex search algorithm (VSA) perform the same task for c...

Driver fatigue is a major cause of traffic accidents. Electroencephalogram (EEG) is considered one of the most reliable predictors of fatigue. This paper proposes a novel, simple and fast method for driver fatigue detection that can be implemented in real-time by using a single-channel on the scalp. The study has two objectives. The first consists of determining the single most relevant EEG channel to monitor fatigue. This is done using maximum covariance analysis. The second objective consists in developing a deep learning method to detect fatigue from this single channel. For this purpose, spectral features of the signal are first extracted. The sequence of features is used to train a Long Short Term Memory (LSTM), deep learning model, to detect fatigue states. Experiments with 12 EEG signals were conducted to discriminate the fatigue stage from the alert stage. Results showed that TP7 was the most significant channel, which is located in the left tempo-parietal region. A zone associated with spatial awareness, visual-spatial navigation, and the cautiousness faculty. In addition, despite the small dataset, the proposed method predicts fatigue with 75% accuracy and a 1.4-second delay. These promising results provide new insights into relevant data for monitoring driver fatigue.

Object detection is a computer vision based technique which is used to detect instances of semantic objects of a particular class in digital images and videos. Crowd density analysis is one of the commonly utilized applications of object detection. Since crowd density classification techniques face challenges like non-uniform density, occlusion, inter-scene, and intra-scene deviations, convolutional neural network (CNN) models are useful. This paper presents a Metaheuristics with Deep Transfer Learning Enabled Intelligent Crowd Density Detection and Classification (MDTL-ICDDC) model for video surveillance systems. The proposed MDTL-ICDDC technique mostly concentrates on the effective identification and classification of crowd density on video surveillance systems. In order to achieve this, the MDTL-ICDDC model primarily leverages a Salp Swarm Algorithm (SSA) with NASNetLarge model as a feature extraction in which the hyperparameter tuning process is performed by the SSA. Furthermore...

Object detection is a computer vision based technique which is used to detect instances of semantic objects of a particular class in digital images and videos. Crowd density analysis is one of the commonly utilized applications of object detection. Since crowd density classification techniques face challenges like non-uniform density, occlusion, inter-scene, and intra-scene deviations, convolutional neural network (CNN) models are useful. This paper presents a Metaheuristics with Deep Transfer Learning Enabled Intelligent Crowd Density Detection and Classification (MDTL-ICDDC) model for video surveillance systems. The proposed MDTL-ICDDC technique mostly concentrates on the effective identification and classification of crowd density on video surveillance systems. In order to achieve this, the MDTL-ICDDC model primarily leverages a Salp Swarm Algorithm (SSA) with NASNetLarge model as a feature extraction in which the hyperparameter tuning process is performed by the SSA. Furthermore...

Permeability is one of the most important rock parameters in reservoir engineering that affects fluids flow in reservoir. In most reservoirs, permeability measurements are rare and Permeability is determined from rock sample or well testing data. Core analysis and well test data are expensive and time consuming. In the present paper, the soft sensor based on a feed-forward artificial neural network (ANN) to estimate permeability of the reservoir is proposed. After that, ANN-based Soft-Sensor was optimized by Shuffled Frog-Leaping Algorithm (SFLA). SFLA is used to decide the initial weights of the neural network. The SFLA-ANN based soft sensor is applied to predict permeability in one of the northern Persian Gulf oil fields of Iran reservoir located in Ahwaz, Iran utilizing available geophysical well log data. The performance of the SFLA-ANN based soft sensor is compared with ANN based soft sensor. The results demonstrate the effectiveness of the SFLA-ANN based soft sensor. (Mohammad...

Permeability is one of the most important rock parameters in reservoir engineering that affects fluids flow in reservoir. In most reservoirs, permeability measurements are rare and Permeability is determined from rock sample or well testing data. Core analysis and well test data are expensive and time consuming. In the present paper, the soft sensor based on a feed-forward artificial neural network (ANN) to estimate permeability of the reservoir is proposed. After that, ANN-based Soft-Sensor was optimized by Shuffled Frog-Leaping Algorithm (SFLA). SFLA is used to decide the initial weights of the neural network. The SFLA-ANN based soft sensor is applied to predict permeability in one of the northern Persian Gulf oil fields of Iran reservoir located in Ahwaz, Iran utilizing available geophysical well log data. The performance of the SFLA-ANN based soft sensor is compared with ANN based soft sensor. The results demonstrate the effectiveness of the SFLA-ANN based soft sensor. (Mohammad...

Energy-efficient buildings have attracted vast attention as a key component of sustainable development. Thermal load analysis is a pivotal step for the proper design of heating, ventilation, and air conditioning (HVAC) systems for increasing thermal comfort in energy-efficient buildings. In this work, novel a methodology is proposed to predict the cooling load (LC) of residential buildings based on their geometrical characteristics. Multi-layer perceptron (MLP) neural network was coupled with metaheuristic algorithms to attain its optimum hyperparameter values. According to the results, the LC pattern can be promisingly captured and predicted by all developed hybrid models. Nevertheless, the comparison analysis revealed that the electrostatic discharge algorithm (ESDA) achieved the most powerful MLP model. Hence, utilizing the proposed methodology would give new insights into the thermal load analysis method and bridge the existing gap between the most recently developed computation...

Estimating the mechanical parameters of concrete is significant towards achieving an efficient mixture design. This research deals with concrete slump analysis using novel integrated models. To this end, four wise metaheuristic techniques of biogeography-based optimization (BBO), salp swarm algorithm (SSA), moth-flame optimization (MFO), and wind driven optimization (WDO) are employed to optimize a popular member of the neural computing family, namely multilayer perceptron (MLP). Four predictive ensembles are constructed to analyze the relationship between concrete slump and seven concrete ingredients including cement, water, slag, fly ash, fine aggregate, superplasticizer, and coarse aggregate. After discovering the optimal complexities by sensitivity analysis, the results demonstrated that the combination of metaheuristic algorithms and neural methods can properly handle the early prediction of concrete slump. Moreover, referring to the calculated ranking scores (RSs), the BBO-MLP...

Object detection is a computer vision based technique which is used to detect instances of semantic objects of a particular class in digital images and videos. Crowd density analysis is one of the commonly utilized applications of object detection. Since crowd density classification techniques face challenges like non-uniform density, occlusion, inter-scene, and intra-scene deviations, convolutional neural network (CNN) models are useful. This paper presents a Metaheuristics with Deep Transfer Learning Enabled Intelligent Crowd Density Detection and Classification (MDTL-ICDDC) model for video surveillance systems. The proposed MDTL-ICDDC technique mostly concentrates on the effective identification and classification of crowd density on video surveillance systems. In order to achieve this, the MDTL-ICDDC model primarily leverages a Salp Swarm Algorithm (SSA) with NASNetLarge model as a feature extraction in which the hyperparameter tuning process is performed by the SSA. Furthermore, a weighted extreme learning machine (WELM) method was utilized for crowd density and classification process. Finally, the krill swarm algorithm (KSA) is applied for an effective parameter optimization process and thereby improves the classification results. The experimental validation of the MDTL-ICDDC approach was carried out with a benchmark dataset, and the outcomes are examined under several aspects. The experimental values indicated that the MDTL-ICDDC system has accomplished enhanced performance over other models such as Gabor, BoW-SRP, Bow-LBP, GLCM-SVM, GoogleNet, and VGGNet.

In recent years, many countries have established their ambitious renewable energy targets to satisfy their future electricity demand with the main aim to foster sustainable and low-emission development. In meeting these targets, the changes to power system planning and operations involve the significant consideration of renewable energy generation through mainly wind energy and solar energy which are more variable and uncertain as compared to the conventional sources (i.e. Thermal and Nuclear Energy). In the present paper, three machine learning methods named as Support Vector Machine, Artificial Neural Network and Multiple Linear Regression are applied to forecast wind power generation on basis of past data of wind direction and wind speed. The impact of input variables such as wind speed and wind direction on wind power generation is investigated and compared.

In this paper, Shuffled Frog Leaping Algorithm is used to improve the recognition rate of Persian handwritten digits. In proposed approach, the effective features in increasing the recognition rate are selected using the Binary Shuffled Frog Leaping Algorithm (BSFLA). By selecting the most suitable features from among all extracted features, the recognition rate is improved and computational costs are also decreased. The fitness function in BSFLA is the number of errors in the Fuzzy classifier which its minimum value is desired. The results indicate that Shuffled Frog Leaping algorithm (SFLA) is more efficient.

Estimating the mechanical parameters of concrete is significant towards achieving an efficient mixture design. This research deals with concrete slump analysis using novel integrated models. To this end, four wise metaheuristic techniques of biogeography-based optimization (BBO), salp swarm algorithm (SSA), moth-flame optimization (MFO), and wind driven optimization (WDO) are employed to optimize a popular member of the neural computing family, namely multilayer perceptron (MLP). Four predictive ensembles are constructed to analyze the relationship between concrete slump and seven concrete ingredients including cement, water, slag, fly ash, fine aggregate, superplasticizer, and coarse aggregate. After discovering the optimal complexities by sensitivity analysis, the results demonstrated that the combination of metaheuristic algorithms and neural methods can properly handle the early prediction of concrete slump. Moreover, referring to the calculated ranking scores (RSs), the BBO-MLP...

This paper presents a physics-inspired metaheuristic optimization algorithm, known as Electromagnetic Field Optimization (EFO). The proposed algorithm is inspired by the behavior of electromagnets with different polarities and takes advantage of a nature-inspired ratio, known as the golden ratio. In EFO, a possible solution is an electromagnetic particle made of electromagnets, and the number of electromagnets is determined by the number of variables of the optimization problem. EFO is a populationbased algorithm in which the population is divided into three fields (positive, negative, and neutral); attraction-repulsion forces among electromagnets of these three fields lead particles toward global minima. The golden ratio determines the ratio between attraction and repulsion forces to help particles converge quickly and effectively. The experimental results on 30 high dimensional CEC 2014 benchmarks reflect the superiority of EFO in terms of accuracy and convergence speed over other state-of-the-art optimization algorithms.

Accurate prediction of fresh concrete slumps is a complex non-linear problem that depends on several parameters including time, temperature, and shear history. It is also affected by the mixture design and various concrete ingredients. This study investigates the efficiency of three novel integrative approaches for predicting this parameter. To this end, the vortex search algorithm (VSA), multi-verse optimizer (MVO), and shuffled complex evolution (SCE) are used to optimize the configuration of multi-layer perceptron (MLP) neural network. The optimal complexity of each model was appraised via sensitivity analysis. Various statistical metrics revealed that the accuracy of the MLP was increased after coupling it with the above metaheuristic algorithms. Based on the obtained results, the prediction error of the MLP was decreased by up to 17%, 10%, and 33% after applying the VSA, MVO, and SCE, respectively. Moreover, the SCE emerged as the fastest optimizer. Accordingly, the novel expli...

Present paper conducts the study on metaheuristic jellyfish optimization algorithm and reviews work done on it. This is a newly developed meta-heuristic optimization algorithm concentrates on movements of jellyfish in ocean such as search of food in sea current, motion inside swarm, bloom, etc. The optimization demonstrates higher and additional favourable results versus other bio-inspired optimization algorithms. Some of the work is discussed in this paper are: finding of unknown parameters of Photo Voltic models using premature convergence strategy, finding of optimal VAR coordination in Automated Delivery System, estimating the parameters of single-phase power transformers, overcoming the drawbacks and improving the quality of network service, tracking the overall maximum power point under partial shade conditions, improving MapReduce job performance and increasing resource usage in Hadoop yarn, solving engineering problems optimally with multiple objectives, dimensionally reduci...

Face recognition is of pronounced significance to real-world applications such as video surveillance systems, human computing interaction, and security systems. This biometric authenticating system encompasses rich real human face characteristics. As such, it has been one of the important research topics in computer vision. Face recognition systems based on deep learning approaches suffer from internal covariate shift problems that cause gradients to explode or gradient disappearance, which leads to improper network training. Improper network training causes network overfitting and computational load. This reduces recognition accuracy and slows down network speed. This paper proposes a modified preactivation batch normalization convolutional neural network by adding a batch normalization layer after each convolutional layer within each of the four convolutional units of the proposed model. The performance of the proposed model is validated with a new dataset, AS-Darmaset, which is b...

Artificial neural networks, especially deep neural networks, are the promising and current research domain as they showed great potential in classification and regression tasks. The process of training artificial neural network (weight optimization), as an NP-hard challenge, is typically performed by backpropagation algorithms such as stochastic gradient descent. However, these types of algorithms are susceptible to trapping the local optimum. Recent studies show that, the metaheuristics-based approaches like swarm intelligence can be efficiently utilized in training the artificial neural network. This paper presents an improved version of swarm intelligence and monarch butterfly optimization algorithm for training the feed-forward artificial neural network. Since the basic monarch butterfly optimization suffers from some deficiencies, improved implementation, that enhances exploration ability and intensification-diversification balance, is devised. Proposed method is validated against 8 well-known classification datasets and compared to similar approaches that were tested within the same environment and simulation setup. Obtained results indicate that, the method proposed in this work outperforms other state-of-the-art algorithms that are shown in the recent outstanding computer science literature.

Many branches of the structural engineering discipline have many problems, which require the generating an optimum model for beam-column junction area reinforcement, weight lightening for members such a beam, column, slab, footing formed as reinforced concrete, steel, composite, and so on, cost arrangement for any construction, etc. With this direction, in the current study, a structural model as a 5-bar truss is handled to provide an optimum design by determining the fittest areas of bar sections. It is aimed that the total bar length is minimized through population-based metaheuristic algorithm as teaching-learning-based optimization (TLBO). Following, the decision-making model is developed via multilayer perceptrons (MLPs) by performing an estimation application to enable directly foreseen of the optimal section areas and total length of bars, besides, the approximation and correlation success are evaluated via some metrics. Thus, determination of the real optimal results of unkn...

In recent years, especially nowadays, artificial intelligence (AI) and machine learning technology that AI's applications are reflected to it are benefited very often in many different fields of study. As the leading cause of this, it can be demonstrated to be able to provide any unknown parameter with higher accuracy, quick as namely effective in terms of consuming time, besides of these, with the usage of less effort and manpower because of the mentioned technologies. In the medical field, diagnosing of disease speedily by be defined of foreign cells; in chemistry or pharmacology, prediction of measures of ingredients within a medicine; or in civil engineering and structural area, detection what properties of materials such as concrete, steel, fly ash, lean concrete that they can ensure of sufficient resistance and safety in any structural member, can be given as examples to this case. Not only mentioned samples, it is possible to find many examples like these in lots of other study fields such as electronics and telecommunication, space researches, language science, and genetics. This technology has an "intelligence," which is artificial, as understood from its name, and this intelligence model was actually put forward by inspiring from humans. On the other side, machine learning is concerned with the usage and progress of

In the last few years, intensive research has been done to enhance artificial intelligence (AI) using optimization techniques. In this paper, we present an extensive review of artificial neural networks (ANNs) based optimization algorithm techniques with some of the famous optimization techniques, e.g., genetic algorithm (GA), particle swarm optimization (PSO), artificial bee colony (ABC), and backtracking search algorithm (BSA) and some modern developed techniques, e.g., the lightning search algorithm (LSA) and whale optimization algorithm (WOA), and many more. The entire set of such techniques is classified as algorithms based on a population where the initial population is randomly created. Input parameters are initialized within the specified range, and they can provide optimal solutions. This paper emphasizes enhancing the neural network via optimization algorithms by manipulating its tuned parameters or training parameters to obtain the best structure network pattern to dissol...

In India, the first confirmed case of novel corona virus (COVID-19) was discovered on 30 January, 2020. The number of confirmed cases is increasing day by day and it crossed 21,53,010 on 09 August, 2020. In this paper a hybrid forecasting model has been proposed to determine the number of confirmed cases for upcoming 10 days based on the earlier confirmed cases found in India. The proposed modelis based on adaptive neuro-fuzzy inference system (ANFIS) and mutation based Bees Algorithm (mBA). ThemetaheuristicBees Algorithm (BA) has been modified applying 4 types of mutation and Mutation based Bees Algorithm (mBA) is applied to enhance the performance of ANFIS by optimizing its parameters. Proposed mBA-ANFIS model has been assessed using COVID-19 outbreak dataset for India and USAand the number of confirmed cases in next 10 days in Indiahas been forecasted. Proposed mBA-ANFIS model has been compared to standard ANFIS model as well as other hybrid models such as GA-ANFIS, DE-ANFIS, HS-...

Face recognition is crucial in real-world applications like video surveillance, human-computer interaction, and security systems. As one of the most important research issues in computer vision, this biometric authenticating system incorporates a wide range of real human facial characteristics. Problems with Internal covariate shift based on deep learning methods for face recognition systems causes gradient explosion or disappearance, resulting in inappropriate network training, network overfitting, and computational load. This lowers recognition accuracy and slows the network speed. Deep learning techniques for face recognition systems must overcome these difficulties. This research presents a modified Pre-activation Batch Normalization Convolutional Neural Network (PABNCNN), which is characteristic with a batch normalization operation after each convolutional layer in all the four convolutional units. The non-Gaussian rectifier linear unit (Relu) activation function works well with this method. The performance of the proposed models is tested using a new dataset called AS-Darmaset, which was created out of the two public online available databases. The two databases are Caltech 101 Objected Categories and Face Recognition Technology (FERET), respectively. This research compared the convergence behavior of the proposed Pre-activation batch Normalization CNN with that of three distinct CNN models. The Post Activation Batch Normalization CNN, Traditional CNN, Sparse Batch Normalization CNN. The experimental results show that the training and validation accuracy of the proposed Pre-activation BN CNN are up to 100.00% and 99.87%. Post Activation Batch Normalization CNN has an accuracy of 100.00% and 99.81% respectively. Traditional CNN has training and validation accuracy of 96.50%, 98.93% and Sparse Normalization CNN has accuracy of 96.50%. CNN has a training accuracy of 96.25% and a validation accuracy of 97.98%.This result illustrates the regularization effect of Pre-Activation-BN-CNN over the state-of-the-art for face recognition systems.

This paper evaluates six soft computational models along with three statistical data-driven models for the prediction of pan evaporation (EP). Accordingly, improved kriging—as a novel statistical model—is proposed for accurate predictions of EP for two meteorological stations in Turkey. In the standard kriging model, the input data nonlinearity effects are increased by using a nonlinear map and transferring input data from a polynomial to an exponential basic function. The accuracy, precision, and over/under prediction tendencies of the response surface method, kriging, improved kriging, multilayer perceptron neural network using the Levenberg–Marquardt (MLP-LM) as well as a conjugate gradient (MLP-CG), radial basis function neural network (RBFNN), multivariate adaptive regression spline (MARS), M5Tree and support vector regression (SVR) were compared. Overall, all the applied models were highly capable of predicting monthly EP in both stations with a mean absolute error (MAE) < ...

This study was carried out to estimate evapotranspiration over the South-Western region of Nigeria, Artificial Neural Network was used for the estimation of Evapotranspiration over South-Western Nigeria. Using a 36 years meteorological data of South-Western Nigeria obtained from NASA (National Aeronautics and Space Administration) Power data, the Multilayer Perceptron Neural Network and Radial Basis Function Neural Network under several Neural Network Architecture was used, training, testing and validation operations also were performed for estimating evapotranspiration closely to the target calculated value. The performance of each neural network under several NN Architecture was evaluated using statistical indicator such as R (Correlation of Coefficient), R2 (Coefficient of Determination), MAE (Mean Absolute Error) and RMSE (Root Mean Square Error). Results present Multilayer Perceptron Neural Network the best neural network with about 70% of its R-values (0.70) because ETo varies...

This article is based on the application of heuristic algorithms to minimize the average power consumption in a VLSI circuit. The idea is to find the optimum layout and temperature for a 3 stage ring oscillator with minimal dynamic average power. The objective function is the same as average power (Pavg) of 3 stage ring oscillator with 6 CMOS inverters that depends on the temperature and the two different group of channel widths for NMOSs and PMOSs. (W1=W3=W5 and W2=W4=W6). These parameters make a three dimensional search space which is explored by search agents of algorithms. Motivated by the convergence of Modified Shuffled Frog Leaping Algorithm (MSFLA), Genetic Algorithm (GA) and the link of MATLAB with HSPICE Software the minimized average power of 3 stage ring oscillator is obtained. Based on MSFLA, Fuzzy-MSFLA, GA, and Fuzzy-GA algorithms the best resulting for Pavg in 0.18µm Technology and the supply voltage of 5v is 1.19 µW based on Fuzzy-MSFLA.

The main objective of this study is to evaluate and compare the performance of different machine learning (ML) algorithms, namely, Artificial Neural Network (ANN), Extreme Learning Machine (ELM), and Boosting Trees (Boosted) algorithms, considering the influence of various training to testing ratios in predicting the soil shear strength, one of the most critical geotechnical engineering properties in civil engineering design and construction. For this aim, a database of 538 soil samples collected from the Long Phu 1 power plant project, Vietnam, was utilized to generate the datasets for the modeling process. Different ratios (i.e., 10/90, 20/80, 30/70, 40/60, 50/50, 60/40, 70/30, 80/20, and 90/10) were used to divide the datasets into the training and testing datasets for the performance assessment of models. Popular statistical indicators, such as Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and Correlation Coefficient (R), were employed to evaluate the predictive cap...

Employing league championship optimization (LCA) technique for adjusting the membership function parameters of the adaptive neuro-fuzzy inference system (ANFIS) is the focal objective of the present study. The mentioned optimization is carried out for better estimation of the soil compression coefficient (SCC) using twelve key factors of soil, namely depth of sample, percentage of sand, percentage of loam, percentage of clay, percentage of moisture content, wet density, dry density, void ratio, liquid limit, plastic limit, plastic Index, and liquidity index. This information is widely useable in designing high-rise buildings located in smart cities. Notably, the used data is collocated from a real-world construction project in Vietnam. The hybrid ensemble of LCA-ANFIS is developed, and the best structure is determined by a three-step sensitivity analysis process. The prediction accuracy of the proposed hybrid model is compared with typical ANFIS to examine the efficiency of the comb...

This paper focuses on the prediction of soil shear strength (SSS), which is one of the most fundamental parameters in geotechnical engineering. Consisting of 12 influential factors, namely depth of sample, percentage of sand, percentage of loam, percentage of clay, percentage of moisture content, wet density, dry density, void ratio, liquid limit, plastic limit, plastic Index, and liquidity index as input variables, as well as the shear strength as the desired output, the dataset is provided through a field survey in Vietnam. Thereafter, as for used intelligent techniques, the main focus of the current study is on evaluating the efficiency of three novel optimization techniques for optimizing an artificial neural network (ANN) in predicting the SSS. To this end, the dragonfly algorithm (DA), whale optimization algorithm (WOA), and invasive weed optimization (IWO) are synthesized with ANN to prevail its computational drawbacks. The complexity of the models is optimized by sensitivity...

In the present work, we employed artificial neural network (ANN) that is optimized with two hybrid models, namely imperialist competition algorithm (ICA) as well as particle swarm optimization (PSO) in the case of the problem of bearing capacity of shallow circular footing systems. Many types of research have shown that ANNs are valuable techniques for estimating the bearing capacity of the soils. However, most ANN training models have some drawbacks. This study aimed to focus on the application of two well-known hybrid ICA–ANN and PSO–ANN models to the estimation of bearing capacity of the circular footing lied in layered soils. In order to provide the training and testing datasets for the predictive network models, extensive finite element (FE) modelling (a database includes 2810 training datasets and 703 testing datasets) are performed on 16 soil layer sets (weaker soil rested on stronger soil and vice versa). Note that all the independent variables of ICA and PSO algorithms are ...

Predicting the level of dissolved oxygen (DO) is an important issue ensuring the sustainability of the inhabitants of a river. A prediction model can predict the DO level using a historical dataset with regard to water temperature, pH, and specific conductance for a given river. The model can be built using sophisticated computational procedures such as multi-layer perceptron-based artificial neural networks. Different types of networks can be constructed for this purpose. In this study, the authors constructed three networks, namely, multi-verse optimizer (MVO), black hole algorithm (BHA), and shuffled complex evolution (SCE). The networks were trained using the datasets collected from the Klamath River Station, Oregon, USA, for the period 2015–2018. We found that the trained networks could predict the DO level of 2019. We also found that both BHA- and SCE-based networks could predict the level of DO using a relatively simple configuration compared to that of MVO. From the viewpoints of absolute errors and Pearson’s correlation coefficient, MVO- and SCE-based networks performed better than BHA-based networks. In synopsis, the authors recommend MVO- and MLP-based artificial neural networks for predicting the DO level of a river. View Full-Text

The notable developments in renewable energy facilities and resources help reduce the cost of production and increase production capacity. Therefore, developers in renewable energy evaluate the overall performance of the various equipment, methods, and structure and then determine the optimal variables for the design of energy production systems. Variables include equipment characteristics and quality, geographical location, and climatic variables such as solar irradiance, temperature, humidity, dust, etc. This paper investigated and reviewed the current big data methods and tools in solar energy production. It discusses the comprehensive two-stage design and evaluation for examining the optimal structure for renewable energy systems. In the design stage, technical and economic aspects are discussed based on a robust analysis of all input/output variables for determining the highest performance. Next, assess and evaluate the effectiveness of each method under different circumstances conditions. Then convert each qualitative indicator into a quantitative measure using extensive data analysis methods to determine the overall performance of the various qualitative variables. The paper also provides an in-depth analysis of the mathematical techniques used in measuring the efficiency of the renewable energy production system and discussing future axes of work in the field of specific energy.

In process industry, liquid flow rate is one of the important variables which need to be controlled to obtain the better quality and reduce the cost of production. The liquid flow rate depends upon number of parameters like sensor output voltage, pipe diameter etc. Conventional approach involves manual tuning of these variables so that optimal flow rate can be achieved which is time consuming and costly. However, estimation of an accurate computational model for liquid flow control process can serve as alternative approach. It is nothing but a non-linear optimization problem. In this work, three different improved versions of original elephant swarm water search algorithm (ESWSA) is proposed and tested against the present problem of liquid flow control. Equations for response surface methodology and analysis of variance are being used as non-linear models and these models are optimized using those newly proposed optimization techniques. The statistical analysis of the obtained results shows that the proposed MESWSA has highest overall efficiency (i.e. 45%) and it outperformed the others techniques for the most of the cases of modeling for liquid flow control process. But one of the major disadvantages of MESWSA is its slow convergence speed. On the other hand, ESWSA is better for finding the best fitness and LESWSA has better stability in output. Moreover, LMESWSA is found to be best efficient algorithm with respect to success rate and computational time. However, all algorithms and models can predict the liquid flow rate with satisfactory accuracy.

A reliable prediction of sustainable energy consumption is key for designing environmentally friendly buildings. In this study, three novel hybrid intelligent methods, namely the grasshopper optimization algorithm (GOA), wind-driven optimization (WDO), and biogeography-based optimization (BBO), are employed to optimize the multitarget prediction of heating loads (HLs) and cooling loads (CLs) in the heating, ventilation and air conditioning (HVAC) systems. Concerning the optimization of the applied algorithms, a series of swarm-based iterations are performed, and the best structure is proposed for each model. The GOA, WDO, and BBO algorithms are mixed with a class of feedforward artificial neural networks (ANNs), which is called a multi-layer perceptron (MLP) to predict the HL and CL. According to the sensitivity analysis, the WDO with swarm size = 500 proposes the most-fitted ANN. The proposed WDO-ANN provided an accurate prediction in terms of heating load (training (R2 correlation = 0.977 and RMSE error = 0.183) and testing (R2 correlation = 0.973 and RMSE error = 0.190)) and yielded the best-fitted prediction in terms of cooling load (training (R2 correlation = 0.99 and RMSE error = 0.147) and testing (R2 correlation = 0.99 and RMSE error = 0.148)).