Intelligent Dendritic Neural Model for Classification Problems

The integration of Fuzzy Neural Networks (FNNs) with optimization techniques has not only solved the issues " black box " in Artificial Neural Networks (ANNs) but also has been effective in a wide variety of real-world applications. Adaptive Neuro-Fuzzy Inference System (ANFIS) still needs effective parameter training and rules optimization methods to perform efficiently when the number of inputs increases. ANFIS accuracy depends on the parameters it is trained with and the drawbacks of gradients based learning of ANFIS using gradient descent and least square methods in two-pass learning algorithm. Many researchers have trained ANFIS parameters using metaheuristic, however, very few have considered optimizing the ANFIS rule-base. We propose an effective technique for optimizing ANFIS rule-base and training the network parameters using newly Accelerated modified MBA (AMBA) to convergence the speed during exploitation phase. The AMBA optimized ANFIS was tested on real-world benchmark classification problems like Breast Cancer, Iris, and Glass. The AMBA optimized ANFIS has also been employed to model real datasets. The performance of the proposed AMBA optimized ANFIS model was compared with the ones optimized by Genetic Algorithm (GA), Particle Swarm Optimization (PSO), MBA and Improved MBA (IMBA), respectively. The results show that the proposed AMBA optimized ANFIS achieved better accuracy with optimized rule-set in less number of function evaluations. Moreover, the results also indicate that AMBA converges earlier than its other counterparts.

IEEE/CAA Journal of Automatica Sinica, 2022

Some recent research reports that a dendritic neuron model (DNM) can achieve better performance than traditional artificial neuron networks (ANNs) on classification, prediction, and other problems when its parameters are well-tuned by a learning algorithm. However, the back-propagation algorithm (BP), as a mostly used learning algorithm, intrinsically suffers from defects of slow convergence and easily dropping into local minima. Therefore, more and more research adopts non-BP learning algorithms to train ANNs. In this paper, a dynamic scale-free network-based differential evolution (DSNDE) is developed by considering the demands of convergent speed and the ability to jump out of local minima. The performance of a DSNDE trained DNM is tested on 14 benchmark datasets and a photovoltaic power forecasting problem. Nine meta-heuristic algorithms are applied into comparison, including the champion of the 2017 IEEE Congress on Evolutionary Computation (CEC2017) benchmark competition effective butterfly optimizer with covariance matrix adapted retreat phase (EBOwithCMAR). The experimental results reveal that DSNDE achieves better performance than its peers.

It is well known that neural networks are a very powerful tool in order to deal with nonlinear and/or adaptive tasks. Nevertheless when faced with nonlinear functions, there is no universal rules to follow in order to build efficiently a good nonlinear classifier. In this context, the paper presents some practical and experience-inspired recommendations in order to be able to appreciate the capacities of the artificial neural network. The way consists in describing the standard and simple methods based on backpropagation learning rule and multilayer neural network, because they are quite simple and they can be applied rapidly to a problem. The first part of the paper is devoted to the presentation of the notation and of the learning rule. The second part presents the neural networks as a nonlinear function builder; it opens a discussion on the key-points to control in order to elaborate a good classifier. The third part details some of the points mentionned before and defines a general canvas helpful for the design of a classifier. The last part illustrates previous considerations on an industrial problem.

Artificial neural networks have been showed their effectiveness in many real world problems such as signal processing, pattern recognition, and classification problems. Although they provide highly generalized solutions, we find several unanswered problems in using artificial neural networks. Determining the most appropriate architecture of artificial neural network is identified as one of those major problems. Generally, the performance of a neural network strongly depends on the size of the network. By increasing the number of layers generalization ability can be improved. However, this solution may not be computationally optimized. On the other hand, too many hidden neurons may over-train the data and which cause the poor generalization. Also, too few neurons under-fit the data and hence, network may not train the data properly. Thus, both too many and too few neurons show bad generalization. Therefore, determining the most suitable architecture is very important in artificial neural networks. As such, a large number of researchers have been carried out to model the hidden layer architecture by using various techniques. These techniques can be categorized as pruning techniques and constructive techniques. Pruning algorithms start with an oversized network and remove nodes until the optimal architecture occurs [1],[2],[3],[4] and [12]. Constructive algorithms [5],[6],[7],[8] do the other way. They build the appropriate neural network during the training process by adding hidden layers, nodes and connection weights to a minimal architecture. However, most of these methods are confined to networks with small number of neurons or single hidden layer neural networks. Hence, they have not addressed the existing problem of hidden layer architecture properly. In this paper, a new pruning algorithm based on backpropagation training [11] has been proposed to design the optimal neural network. The optimal solution is obtained by two steps. First, the number of hidden layers in the most efficient network is determined. Then the network tends to the optimal solution by removing all unimportant nodes from each layer. The removable nodes are identified through the delta values of hidden neurons [9],[10]. The choosing of delta values was based on the fact that the delta values of the hidden layers are used to compute the error term of the next training cycle. Hence, delta value is a significant factor in error term. Thus, the delta values are used to identify the less saliency neurons and remove them from hidden neurons so that the error term tends to the desired limit faster than the backpropagation training. The approaches of the other researchers are discussed in the next section. Section III describes the new algorithm and how to use the delta values in optimization of hidden layer architecture. The experimental method and results are discussed in section IV. Finally, section V presents the conclusions.

American Journal of Physics and Applications, 2014

In a complex and changing a remote sensing system, which requires taking quick and informed decisions environment, connectionist methods have shown their great contribution in particular the reduction and classification of spectral data. In this context, this paper proposes to study the parameters that optimize the results of an artificial neural network ANN multilayer perceptron based, for classification of chemical agents on multi-spectral images. The mean squared error cost function remains one of the major parameters of the network convergence at its learning phase and a challenge that will face our approach to improve the gradient descent by the conjugate gradient method that seems fast and efficient.

2018 IEEE Congress on Evolutionary Computation (CEC), 2018

Intrusion detection systems are developed with the abilities to discriminate between normal and anomalous traffic behaviours. The core challenge in implementing an intrusion detection systems is to determine and stop anomalous traffic behavior precisely before it causes any adverse effects to the network, information systems, or any other hardware and digital assets which forming or in the cyberspace. Inspired by the biological immune system, Dendritic Cell Algorithm (DCA) is a classification algorithm developed for the purpose of anomaly detection based on the danger theory and the functioning of human immune dendritic cells. In its core operation, DCA uses a weighted sum function to derive the output cumulative values from the input signals. The weights used in this function are either derived empirically from the data or defined by users. Due to this, the algorithm opens the doors for users to specify the weights that may not produce optimal result (often accuracy). This paper proposes a weight optimisation approach implemented using the popular stochastic search tool, genetic algorithm. The approach is validated and evaluated using the KDD99 dataset with promising results generated.

Neurocomputing, 2010

The use of artificial neural networks implies considerable time spent choosing a set of parameters that contribute toward improving the final performance. Initial weights, the amount of hidden nodes and layers, training algorithm rates and transfer functions are normally selected through a manual process of trial-and-error that often fails to find the best possible set of neural network parameters for a specific problem. This paper proposes an automatic search methodology for the optimization of the parameters and performance of neural networks relying on use of Evolution Strategies, Particle Swarm Optimization and concepts from Genetic Algorithms corresponding to the hybrid and global search module. There is also a module that refers to local searches, including the well-known Multilayer Perceptrons, Back-propagation and the Levenberg-Marquardt training algorithms. The methodology proposed here performs the search using the aforementioned parameters in an attempt to optimize the networks and performance. Experiments were performed and the results proved the proposed method to be better than trial-and-error and other methods found in the literature.

IEEE Access

Chemical reaction optimization (CRO) algorithm is a robust metaheuristic by simulating the process of natural chemical reaction and has been applied for solving numerous realistic problems. However, it is suffering from low population diversity, slow convergence, and deprived from local search ability. To boost the search operation of CRO, this article proposes an improved CRO method termed as ICRO. We achieve the improvements in two phases. First, to start the search operation with a better-quality population, we propose a new method to initialize population which helps in enhancing diversification of the search space. Second, to intensify the local search ability we integrate Nelder-Mead simplex method with CRO. The ICRO is then evaluated on a few benchmark functions for optimization and found apposite. Dendritic neuron model (DNM) using additive and multiplicative-based aggregation functions has been emerging as a machine learning approach and found successful in many engineering applications. This study attempts to advance the predictive accuracy of DNM through maintaining a decent steadiness between exploitation and exploration of its search space with the proposed ICRO, termed as ICRO-DNM. The powerful global search ability of ICRO synergies with better approximation capability of DNM thus, able to overcome the limitations of conventional back propagation learning based DNM. ICRO-DNM is evaluated on closing price prediction for four stock index and net asset values of four mutual fund datasets and found efficient. The consequences of this attempt are: 1) Proposed ICRO shows robust parameter optimization ability for benchmark functions and DNM compared to basic CRO, particle swarm optimization (PSO), and genetic algorithm (GA) and 2) The learning paradigm formed due to reasonable amalgamation of ICRO and DNM is pretty able to capture the underlying uncertainties coupled with financial data, produces more precise and steady predictions and significantly different from other forecasts. INDEX TERMS Applications of artificial intelligence, nature inspired optimization, chemical reaction optimization, dendritic neuron model, financial time series forecasting.

Neural Net Classifier (NNC). The objective is to compare the performances of BP, SA, and BCGA in terms of yielding accurate outputs in less time. Four standard datasets, e.g. Iris, Diabetes, Glass and Teaching Assistant Evaluation (TAE) are used for their implementations and testing. It has been observed that BP loses its merit as a good NNC optimizer due to its tendency of being trapped into local minima. SA, on the other hand, consumes more time for convergence, but yields better classification accuracy, compared to BP. BCGA, being a global search algorithm, is able to find better solution from all available solutions and thus yields the best accuracy, but with slow convergence rate. Based on these observations, the paper suggests that SA might be a reasonable choice for NNC optimization to start with, when both accuracy and convergence speed are considered.

IEEE Transactions on Neural Networks, 2006

This paper introduces a methodology for neural network global optimization. The aim is the simultaneous optimization of multilayer perceptron (MLP) network weights and architectures, in order to generate topologies with few connections and high classification performance for any data sets. The approach combines the advantages of simulated annealing, tabu search and the backpropagation training algorithm in order to generate an automatic process for producing networks with high classification performance and low complexity. Experimental results obtained with four classification problems and one prediction problem has shown to be better than those obtained by the most commonly used optimization techniques. Index Terms-Multilayer perceptron (MLP), optimization of weights and architectures, simulating annealing, tabu search.