Optimal transfer function neural networks

Neural Computing Surveys, 1999

The choice of transfer functions may strongly influence complexity and performance of neural networks. Although sigmoidal transfer functions are the most common there is no a priori reason why models based on such functions should always provide optimal decision borders. A large number of alternative transfer functions has been described in the literature. A taxonomy of activation and output functions is proposed, and advantages of various non-local and local neural transfer functions are discussed. Several less-known types of transfer functions and new combinations of activation/output functions are described. Universal transfer functions, parametrized to change from localized to delocalized type, are of greatest interest. Other types of neural transfer functions discussed here include functions with activations based on non-Euclidean distance measures, bicentral functions, formed from products or linear combinations of pairs of sigmoids, and extensions of such functions making rotations of localized decision borders in highly dimensional spaces practical. Nonlinear input preprocessing techniques are briefly described, offering an alternative way to change the shapes of decision borders.

Automatic Learning Techniques in Power Systems, 1998

2017

Neural Networks (NN) is an effective approach used in many fields such as medicine, industry, security, stock market prediction, character recognition, image processing and many other fields. The main problem in the use of Artificial Neural Networks (ANN) is the control of parameters, since there is no explicit and specific way to determine the optimal values of Neural Network parameters. The aim of this study is to choose the best parameters that can be used to model and optimize the number of hidden layers, number of neurons contained in there, momentum, delta rule, transition functions and multidimensional network.

… 2000. IJCNN 2000, Proceedings of the …, 2000

The choice of transfer functions may strongly influence complexity and performance of neural networks used in classification and approximation tasks. A taxonomy of activation and output functions is proposed, allowing to generate many transfer functions. Several less-known types of transfer functions and new combinations of activation/output functions are described. Functions parameterize to change from localized to delocalized type, functions with activation based on non-Euclidean distance measures, bicentral functions formed from pairs of sigmoids are discussed.

Neural Processing Letters, 2009

This work analyzes the problem of selecting an adequate neural network architecture for a given function, comparing existing approaches and introducing a new one based on the use of the complexity of the function under analysis. Numerical simulations using a large set of Boolean functions are carried out and a comparative analysis of the results is done according to the architectures that the different techniques suggest and based on the generalization ability obtained in each case. The results show that a procedure that utilizes the complexity of the function can help to achieve almost optimal results despite the fact that some variability exists for the generalization ability of similar complexity classes of functions.

ACM SIGAda Ada Letters, 1994

A neural network is a computer program structured as a simplified model of a brain . It contains nodes (analogous to neurons) and connections between node s (analogous to synapses) . Neural networks can solve difficult pattern-matchin g problems . A node sums the inputs it receives from other nodes and passes the result through a transfer function to produce its output . A modifiable weight i s associated with each connection . A network is trained on a given training set o f inputs. During training, the weights are successively adjusted to produce th e desired output.

2014

Prajna Behera Graduate Student IIT Kanpur Abstarct Neural network is one of the important components in Artificial Intelligence (AI). It is a computational model for storing and retrieving acquired knowledge .It has been studied for many years in the hope of achieving human-like performance in many fields, such as speech and image recognition as well as information retrieval. ANNs consists of dense interconnected computing units that are simple models for complex neurons in biological systems. The knowledge is acquired during a learning process and is stored in the synaptic weights of the inter-nodal connections. The main advantage of neural network is their ability to represent complex input/output relationships. The performance of an ANN is measured according to an error between target and actual output, complexity of ANN, training time etc. The topologicalstructure of an ANN can be characterized by the arrangement of the layers and neurons, the nodal connectivity, and the nodal t...

2009 IEEE International Conference on Industrial Technology, 2009

Modeling helps to understand and predict the outcome of complex systems. Inductive modeling methodologies are beneficial for modeling the systems where the uncertainties involved in the system do not permit to obtain an accurate physical model. However inductive models, like Artificial Neural Networks (ANNs), may suffer from a few drawbacks involving over-fitting and the difficulty to easily understand the model itself. This can result in user reluctance to accept the model or even complete rejection of the modeling results. Thus, it becomes highly desirable to make such inductive models more comprehensible and to automatically determine the model complexity to avoid over-fitting. In this paper, we propose a novel type of ANN, a Mixed Transfer Function Artificial Neural Network (MTFANN), which aims to improve the complexity fitting and comprehensibility of the most popular type of ANN (MLP -a Multilayer Perceptron).