Papers by Behzad Behzadan
This article presents a new approach for automatically determining the optimal quantity and conne... more This article presents a new approach for automatically determining the optimal quantity and connectivity of the hidden-layer of a three-layer Feed-Forward Neural Network (FFNN) based on a theoretical and practical approach. The system (MINES) is a combination of Neural Network (NN), Back-Propagation (BP), Genetic Algorithm (GA), Mutual Information (MI), and clustering. BP is used to reduce the training-error while MI aides BP to follow an effective path. A GA changes the incoming synaptic connections of the hidden-nodes based on MI fitness. Assigning MI as the fitness of individuals brings a competition between hidden-nodes to acquire a higher amount of information from the error-space. Weight clustering is applied to reduce those hidden-nodes having similar weights. Experimental results are presented, and future directions discussed.
We present a practical approach towards neural network structural learning which is theoretically... more We present a practical approach towards neural network structural learning which is theoretically founded on information theory. A novel self-governing system, called Mutual information neuro-evolutionary system (MINES), is introduced with the ability of determining the optimal quantity and connectivity of the hidden layer of a three layer feedforward neural network. The system is a combination of a feed-forward neural network, back-propagation algorithm, genetic algorithm, mutual information and clustering. Backpropagation is used for parameter learning to reduce the system's error; while mutual information aids back-propagation to follow an effective path in the weight space. A genetic algorithm changes the incoming synaptic connections of the hidden nodes, based on the fitness provided by the mutual information from the error space to the hidden layer, to perform structural learning. Mutual information, between node output and network error is used as the GA fitness function for nodes. Weight clustering is applied to reduce hidden nodes having similar functionality; i.e. those possessing same connectivity patterns and close Euclidean angle in the weight space. Finally, the performance of the system is assessed on two theoretical and one empirical problems.
Uploads
Papers by Behzad Behzadan