Academia.eduAcademia.edu

Outline

Title
Abstract
Experimental Results
Discussion and Future Research
References
All Topics
Computer Science
Machine Learning

Very Fast Online Learning of Highly Non Linear Problems

Profile image of Aggelos ChariatisAggelos Chariatis

2007, Journal of Machine Learning Research

Abstract

The experimental investigation on the efficient learning of highly non-linear problems by online training, using ordinary feed forward neural networks and stochastic gradient descent on the errors computed by back-propagation, gives evidence that the most crucial factors for efficient training are the hidden units' differentiation, the attenuation of the hidden units' interference and the selective attention on the parts of the problems where the approximation error remains high. In this report, we present global and local selective attention techniques and a new hybrid activation function that enables the hidden units to acquire individual receptive fields which may be global or local depending on the problem's local complexities. The presented techniques enable very efficient training on complex classification problems with embedded subproblems.

Related papers

A Neural Network Approach for Online Nonlinear Neyman-Pearson Classification
Basarbatu Can (Student)

IEEE Access

We propose a novel Neyman-Pearson (NP) classifier that is both online and nonlinear as the first time in the literature. The proposed classifier operates on a binary labeled data stream in an online manner, and maximizes the detection power about a user-specified and controllable false positive rate. Our NP classifier is a single hidden layer feedforward neural network (SLFN), which is initialized with random Fourier features (RFFs) to construct the kernel space of the radial basis function at its hidden layer with sinusoidal activation. Not only does this use of RFFs provide an excellent initialization with great nonlinear modeling capability, but it also exponentially reduces the parameter complexity and compactifies the network to mitigate overfitting while improving the processing efficiency substantially. We sequentially learn the SLFN with stochastic gradient descent updates based on a Lagrangian NP objective. As a result, we obtain an expedited online adaptation and powerful nonlinear Neyman-Pearson modeling. Our algorithm is appropriate for large scale data applications and provides a decent false positive rate controllability with real time processing since it only has O(N) computational and O(1) space complexity (N : number of data instances). In our extensive set of experiments on several real datasets, our algorithm is highly superior over the competing state-of-the-art techniques, either by outperforming in terms of the NP classification objective with a comparable computational as well as space complexity or by achieving a comparable performance with significantly lower complexity.

View PDFchevron_right
On-line learning algorithms for locally recurrent neural networks
Aurelio Uncini

IEEE Transactions on Neural Networks, 1999

This paper focuses on on-line learning procedures for locally recurrent neural networks with emphasis on multilayer perceptron (MLP) with infinite impulse response (IIR) synapses and its variations which include generalized output and activation feedback multilayer networks (MLN's). We propose a new gradient-based procedure called recursive backpropagation (RBP) whose on-line version, causal recursive backpropagation (CRBP), presents some advantages with respect to the other on-line training methods. The new CRBP algorithm includes as particular cases backpropagation (BP), temporal backpropagation (TBP), backpropagation for sequences (BPS), Back-Tsoi algorithm among others, thereby providing a unifying view on gradient calculation techniques for recurrent networks with local feedback. The only learning method that has been proposed for locally recurrent networks with no architectural restriction is the one by Back and Tsoi. The proposed algorithm has better stability and higher speed of convergence with respect to the Back-Tsoi algorithm, which is supported by the theoretical development and confirmed by simulations. The computational complexity of the CRBP is comparable with that of the Back-Tsoi algorithm, e.g., less that a factor of 1.5 for usual architectures and parameter settings. The superior performance of the new algorithm, however, easily justifies this small increase in computational burden. In addition, the general paradigms of truncated BPTT and RTRL are applied to networks with local feedback and compared with the new CRBP method. The simulations show that CRBP exhibits similar performances and the detailed analysis of complexity reveals that CRBP is much simpler and easier to implement, e.g., CRBP is local in space and in time while RTRL is not local in space.

View PDFchevron_right
Fast and Efficient Learning Algorithms for Feed-forward Neural Networks
Nicolaos Karayiannis

2019

This paper presents a generalized criterion proposed for training feed-forward neural networks and evaluates the performance of the resulting gradient-descent-based learning algorithm. This paper also presents revisions, improvements, and extensions of the family of ELEANNE algorithms, proposed for training multilayered neural networks. The comparison of the proposed with their original version and gradient-descent-based algorithms indicated that the proposed revisions of the algorithms improve significantly their convergence, reduce the occurrence of local minima during the training process, and make the algorithms less sensitive to the parameters provided by the user.

View PDFchevron_right
Special issue on machine learning for nonlinear processing
Carlos M. Travieso

Neurocomputing, 2014

View PDFchevron_right
On-line learning with minimal degradation in feedforward networks
C. Torras

IEEE Transactions on Neural Networks, 1995

Dealing with non-stationary processes requires quick adaptation while at the same time avoiding catastrophic forgetting. A neural learning technique that satisfies these requirements, without sacrifying the benefits of distributed representations, is presented. It relies on a formalization of the problem as the minimization of the error over the previously learned input-output (i-o) patterns, subject to the constraint of perfect encoding of the new pattern. Then this constrained optimization problem is transformed into an unconstrained one with hidden-unit activations as variables. This new formulation naturally leads to an algorithm for solving the problem, which we call Learning with Minimal Degradation (LMD). Some experimental comparisons of the performance of LMD with back-propagation are provided which, besides showing the advantages of using LMD, reveal the dependence of forgetting on the learning rate in back-propagation. We also explain why overtraining affects forgetting and faulttolerance, which are seen as related problems.

View PDFchevron_right
On-line training of recurrent neural networks with continuous topology adaptation
Dragan Obradovic

IEEE Transactions on Neural Networks, 1996

View PDFchevron_right
Globally Optimal Parameters for On-Line Learning in Multilayer Neural Networks
David Saad

Physical Review Letters, 1997

We present a framework for calculating globally optimal parameters, within a given time frame, for on-line learning in multilayer neural networks. We demonstrate the capability of this method by computing optimal learning rates in typical learning scenarios. A similar treatment allows one to determine the relevance of related training algorithms based on modi cations to the basic gradient descent rule as well as to compare di erent training methods.

View PDFchevron_right
An Efficient Gradient-based Learning Algorithm Applied to Neural Networks with Selective Actuation Neurons
Noel Lopes, Bernardete Ribeiro

Neural Parallel and Scientific Computations, 2003

A new class of Neural Networks (NN), designated the Multiple Feed-Forward (MFF) networks, and a new gradient-based learning algorithm, Multiple Back-Propagation (MBP), are proposed and analyzed. MFF are obtained by integrating two feed-forward networks (a main network and a space network) in a novel manner. A major characteristic is their ability to partition the input space by using selective neurons, whose actuation role is captured through the space localisation of input pattern data. In this sense, only those neurons fired by a particular data point turn out to be relevant, while they retain the capacity to approximate closely to more general, irregular, non-linear features in localized regions. Together, the MFF networks and the MBP algorithm embody a new neural architecture, ensuring, in most cases, a better design choice than the one provided by the Multi-Layer Perceptron (MLP) networks trained with the Back-Propagation (BP) algorithm. The utilization of computable importance factors for the actuation neurons whose relative magnitudes are derived from the space network properties and the training data is the key reason for its ability to decompose the underlying mapping function into simpler sub-functions requiring parsimonious NN. Experimental results on benchmarks confirm improved efficiency of the gradient-based learning algorithm proposed, borne out by better generalization and in most cases by shorter training times for online learning, as compared with the MLP networks trained with the BP algorithm. the networks is responsible for determining the relevance of the other network's neurons, as far as each pattern is submitted to the MFF network. By selectively choosing the importance of each neuron, according to the pattern presented, the MFF network is implicitly empowered with the ability to divide the input space, assigning a different network to each sub-space. The two networks encompassing the MFF network work cooperatively, as a unit and therefore must be trained together. This can be accomplished by a gradient-based algorithm, specifically developed to train MFF networks, designated the MBP algorithm. The MFF networks trained with this algorithm offer advantages over the MLP networks trained by the BP algorithm: (i ) better generalization capabilities, and (ii ) shorter training times for the online training mode . Experimental results that support these advantages are presented and compared below. This paper is organized as follows. Section 2 describes the MFF networks, while Section 3 is devoted to the detailed description of MBP algorithm. Results for benchmark data sets, in both classification and regression tasks, are set forth and discussed in section 4. Finally, section 5 presents the conclusions.

View PDFchevron_right
Online Learning From Finite Training Sets and
Peter Sollich

We analyse online gradient descent learning from nite training sets at non-in nitesimal learning rates for both linear and non-linear networks. In the linear case, exact results are obtained for the time-dependent generalization error of networks with a large number of weights N, trained on p = N examples. This allows us to study in detail the e ects of nite training set size on, for example, the optimal choice of learning rate. We also compare online and o ine learning, for respective optimal settings of at given nal learning time. Online learning turns out to be much more robust to input bias and actually outperforms o ine learning when such bias is present; for unbiased inputs, online and o ine learning perform almost equally well. Our analysis of online learning for non-linear networks (namely, softcommittee machines), advances the theory to more realistic learning scenarios. Dynamical equations are derived for an appropriate set of order parameters; these are exact in the limiting case of either linear networks or in nite training sets. Preliminary comparisons with simulations suggest that the theory captures some e ects of nite training sets, but may not yet account correctly for the presence of local minima.

View PDFchevron_right
A hybrid linear/nonlinear training algorithm for feedforward neural networks
Gordon Lightbody

IEEE Transactions on Neural Networks, 1998

This paper presents a new hybrid optimization strategy for training feedforward neural networks. The algorithm combines gradient-based optimization of nonlinear weights with singular value decomposition (SVD) computation of linear weights in one integrated routine. It is described for the multilayer perceptron (MLP) and radial basis function (RBF) networks and then extended to the local model network (LMN), a new feedforward structure in which a global nonlinear model is constructed from a set of locally valid submodels. Simulation results are presented demonstrating the superiority of the new hybrid training scheme compared to second-order gradient methods. It is particularly effective for the LMN architecture where the linear to nonlinear parameter ratio is large.

View PDFchevron_right

References (48)

  1. C. C. Aggarwal, A. Hinneburg, and D. A. Keim. On the surprising behavior of distance metrics in high dimensional spaces. In J. Van den Bussche and V. Vianu, editors, Proceedings of the 8th International Conference on Database Theory (ICDT), volume 1973 of Lecture Notes in Computer Science, pages 420-434. Springer, 2001.
  2. K. Agyepong and R. Kothari. Controlling hidden layer capacity through lateral connections. Neural Computation, 9(6):1381-1402, 1997.
  3. S. Ahmad and S. Omohundro. A network for extracting the locations of point clusters using selective attention. In Proceedings of the 12th Annual Conference of the Cognitive Science Society, MIT, 1990.
  4. L. B. Almeida, T. Langlois, and J. D. Amaral. On-line step size adaptation. Technical Report INESC RT07/97, INESC/IST, Rua Alves Redol 1000 Lisbon, Portugal, 1997.
  5. S. Amari. Natural gradient works efficiently in learning. Neural Computation, 10(2):251-276, 1998.
  6. P. Bakker. Don't care margins help backpropagation learn exceptions. In A. Adams and L. Sterling, editors, Proceedings of the 5th Australian Joint Conference on Artificial Intelligence, pages 139- 144, 1992.
  7. P. Bakker. Exception learning by backpropagation: A new error function. In P. Leong and M. Jabri, editors, Proceedings of the 4th Australian Conference on Neural Networks, pages 118-121, 1993.
  8. S. Baluja and D. Pomerleau. Using the representation in a neural network's hidden layer for task- specific focus of attention. In IJCAI, pages 133-141, 1995.
  9. L. Breiman. Bias, variance, and arcing classifiers. Technical Report 460, Statistics Department, University of California, 1996.
  10. D. S. Broomhead and D. Lowe. Multivariate functional interpolation and adaptive networks. Com- plex Systems, 2(3):321-355, 1988.
  11. W. Duch, K. Grudzinski, and G. H. F. Diercksen. Minimal distance neural methods. In World Congress of Computational Intelligence, pages 1299-1304, 1998.
  12. D. L. Elliott. A better activation function for artificial neural networks. Technical Report TR 93-8, The Institute for Systems Research, University of Maryland, College Park, MD, 1993.
  13. G. W. Flake. Square unit augmented, radially extended, multilayer perceptrons. In G. B. Orr and K. R. Müller, editors, Neural Networks: Tricks of the Trade, volume 1524 of Lecture Notes in Computer Science, pages 145-163. Springer, 1998.
  14. T. C. Fogarty. Technical note: First nearest neighbor classification on frey and slate's letter recog- nition problem. Machine Learning, 9(4):387-388, 1992.
  15. Y. Freund and R. E. Schapire. Experiments with a new boosting algorithm. In ICML, pages 148- 156, 1996.
  16. P. W. Frey and D. J. Slate. Letter recognition using holland-style adaptive classifiers. Machine Learning, 6:161-182, 1991.
  17. S. Geman and D. Geman. Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(6):721-741, 1984.
  18. T. Graepel and N. N. Schraudolph. Stable adaptive momentum for rapid online learning in nonlinear systems. In J. R. Dorronsoro, editor, Proceedings of the International Conference on Artificial Neural Networks (ICANN), volume 2415 of Lecture Notes in Computer Science, pages 450-455. Springer, 2002.
  19. M. Harmon and L. Baird. Multi-player residual advantage learning with general function approx- imation. Technical Report WL-TR-1065, Wright Laboratory, Wright-Patterson Air Force Base, OH 45433-6543, 1996.
  20. M. Hegland and V. Pestov. Additive models in high dimensions. Computing Research Repository (CoRR), cs/9912020, 1999.
  21. S. C. Huang and Y. F. Huang. Learning algorithms for perceptrons using back propagation with selective updates. IEEE Control Systems Magazine, pages 56-61, April 1990.
  22. R.A. Jacobs. Increased rates of convergence through learning rate adaptation. Neural Networks, 1: 295-307, 1988.
  23. R. Kothari and D. Ensley. Decision boundary and generalization performance of feed-forward networks with gaussian lateral connections. In S. K. Rogers, D. B. Fogel, J. C. Bezdek, and B. Bosacchi, editors, Applications and Science of Computational Intelligence, SPIE Proceedings, volume 3390, pages 314-321, 1998.
  24. B. Laheld and J. F. Cardoso. Adaptive source separation with uniform performance. In Proc. EUSIPCO, pages 183-186, September 1994.
  25. Y. LeCun, P. Simard, and B. Pearlmutter. Automatic learning rate maximization by on-line estima- tion of the hessian's eigenvectors. In S. Hanson, J. Cowan, and L. Giles, editors, Advances in Neural Information Processing Systems, volume 5, pages 156-163. Morgan Kaufmann Publish- ers, San Mateo, CA, 1993.
  26. Y. LeCun, L. Bottou, G. B. Orr, and K.-R. Mueller. Efficient backprop. In G. B. Orr and K.-R. Müller, editors, Neural Networks: Tricks of the Trade, volume 1524 of Lecture Notes in Computer Science, pages 9-50. Springer, 1998.
  27. T. K. Leen and G. B. Orr. Optimal stochastic search and adaptive momentum. In J. D. Cowan, G. Tesauro, and J. Alspector, editors, Proceedings of the 7th NIPS Conference (NIPS), Advances in Neural Information Processing Systems 6, pages 477-484. Morgan Kaufmann, 1993.
  28. P. W. Munro. A dual back-propagation scheme for scalar reinforcement learning. In Proceedings of the 9th Annual Conference of the Cognitive Science Society, Seattle, WA, pages 165-176, 1987.
  29. N. Murata, K. Müller, A. Ziehe, and S. Amari. Adaptive on-line learning in changing environments. In M. Mozer, M. I. Jordan, and T. Petsche, editors, Advances in Neural Information Processing Systems 9 (NIPS), pages 599-605. MIT Press, 1996.
  30. D. J. Newman, S. Hettich, C.L. Blake, and C.J. Merz. UCI repository of machine learning databases, 1998.
  31. G. B. Orr and T. K. Leen. Using curvature information for fast stochastic search. In M. Mozer, M. I. Jordan, and T. Petsche, editors, Advances in Neural Information Processing Systems 9 (NIPS), pages 606-612. MIT Press, 1996.
  32. J. L. Phillips and D. C. Noelle. Reinforcement learning of dimensional attention for categorization. In Proceedings of the 26th Annual Meeting of the Cognitive Science Society, 2004.
  33. M. Plumbley. A hebbian/anti-hebbian network which optimizes information capacity by orthonor- malizing the principal subspace. In Proc. IEE Conf. on Artificial Neural Networks, Brighton, UK, pages 86-90, 1993.
  34. R. Reed, R.J. Marks, and S. Oh. Similarities of error regularization, sigmoid gain scaling, target smoothing, and training with jitter. IEEE Transactions on Neural Networks, 6(3):529-538, 1995.
  35. R. E. Schapire. A brief introduction to boosting. In T. Dean, editor, Proceedings of the 16th Interna- tional Joint Conference on Artificial Intelligence (IJCAI), pages 1401-1406. Morgan Kaufmann, 1999.
  36. R. E. Schapire, Y. Freund, P. Barlett, and W. S. Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. In D. H. Fisher, editor, Proceedings of the 14th International Conference on Machine Learning (ICML), pages 322-330. Morgan Kaufmann, 1997.
  37. N. N. Schraudolph. Fast curvature matrix-vector products for second-order gradient descent. Neural Computation, 14(7):1723-1738, 2002.
  38. N. N. Schraudolph. Centering neural network gradient factors. In G. B. Orr and K. R. M üller, edi- tors, Neural Networks: Tricks of the Trade, volume 1524 of Lecture Notes in Computer Science, pages 207-226. Springer, 1998a.
  39. N. N. Schraudolph. Accelerated gradient descent by factor-centering decomposition. Technical Report IDSIA-33-98, Istituto Dalle Molle di Studi sull'Intelligenza Artificiale, 1998b.
  40. N. N. Schraudolph. Online local gain adaptation for multi-layer perceptrons. Technical Report IDSIA-09-98, Istituto Dalle Molle di Studi sull'Intelligenza Artificiale, Galleria 2, CH-6928 Manno, Switzerland, 1998c.
  41. N. N. Schraudolph. Local gain adaptation in stochastic gradient descent. In ICANN, pages 569-574. IEE, London, 1999.
  42. H. Schwenk and Y. Bengio. Boosting neural networks. Neural Computation, 12(8):1869-1887, 2000.
  43. H. Schwenk and Y. Bengio. Training methods for adaptive boosting of neural networks for character recognition. In M. Jordan, M. Kearns, and S. Solla, editors, Advances in Neural Information Processing Systems 10. MIT Press, Cambridge, MA, 1998.
  44. M. W. Spratling and M. H. Johnson. Neural coding strategies and mechanisms of competition. Cognitive Systems Research, 5(2):93-117, 2004.
  45. C. Thornton. The howl effect in dynamic-network learning. In Proceedings of the International Conference on Artificial Neural Networks, pages 211-214, 1992.
  46. K. M. Ting and Z. Zheng. Improving the performance of boosting for naive bayesian classification. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 296-305, 1999.
  47. Y. H. Yu and R. F. Simmons. Descending epsilon in back-propagation: A technique for better gen- eralization. In Proceedings of the International Joint Conference on Neural Networks (IJCNN), volume 3, pages 167-172, 1990.
  48. S. Zhong and J. Ghosh. Decision boundary focused neural network classifier. In Intelligent Engi- neering Systems Through Artificial Neural Networks (ANNIE). ASME Press, 2000.

Related papers

Online sequential learning with non-iterative strategy for feature extraction, classification and data augmentation
Adhri Paul

2020

Network aims to optimize for minimizing the cost function and provide better performance. This experimental optimization procedure is widely recognized as gradient descent, which is a form of iterative learning that starts from a random point on a function and travels down its slope, in steps, until it reaches to the steepest point which is time-consuming and slow to converge. Over the last couple of decades, several variations of the non-iterative neural network training algorithms have been proposed, such as Random Forest and Quicknet. However, the non-iterative neural network training algorithms do not support online training that given a very largesized training data, one needs enormous computing resources to train neural network. In this thesis, a non-iterative learning strategy with online sequential has been exploited. In Chapter 3, a single layer Online Sequential Sub-Network node (OS-SN) classifier has been proposed that can provide competitive accuracy by pulling the resid...

View PDFchevron_right
Non-iterative approaches in training feed-forward neural networks and their applications
Xizhao Wang

Soft Computing, 2018

Focusing on non-iterative approaches in training feed-forward neural networks, this special issue includes 12 papers to share the latest progress, current challenges, and potential applications of this topic. This editorial presents a background of the special issue and a brief introduction to the 12 contributions. Keywords Non-iterative approaches • Feed-forward neural networks • Neural networks with random weights • Deep learning In recent years, artificial neural networks have been widely studied and applied in many areas, especially in deep neural networks which have made many breakthroughs in image processing, speech recognition, natural language processing, etc. The training mechanism of traditional neural networks is that all the internal parameters need to be iteratively finetuned by using a gradient descent technique . During the training process, the derivatives of the loss function are continually back propagated to each hidden layer to guide the parameter adjustment until the difference between the model prediction and real observation is small enough. With the increase in the number of hidden layers and the number of hidden layer nodes, this method may cause problems such as slow convergence rate, time-consuming, local minima problem, and excessive resource consumption. In order to alleviate the above problem, many novel algorithms have been proposed to optimize the training process of traditional neural networks. In this special issue, we focus on non-iterative approaches in training feed-forward neural networks, which include extreme learning machine (ELM) , random vector functional link network (RVFL) (Pao and Takefuji 1992), neural networks with random weights (NNRW) (Schmidt et al. 1992), etc.

View PDFchevron_right
Efficient Training of Feed-Forward Neural Networks
martin møller

DAIMI Report Series, 1993

Since the discovery of the back-propagation method, many modi ed and new algorithms have been proposed for training of feed-forward neural networks. The problem with slow convergence rate has, however, not been solved when the training is on large scale problems. There is still a need for more e cient algorithms. This Ph.D. thesis describes di erent approaches to improve convergence. The main results of the thesis is the development of the Scaled Conjugate Gradient Algorithm and the stochastic version of this algorithm. Other important results are the development of methods that can derive and use Hessian information in an e cient way. The main part of this thesis is the 5 papers presented in appendices A-E. Chapters 1-6 give an overview of learning in feed-forward neural networks, put these papers in perspective and present the most important results. The conclusion of this thesis is: Conjugate gradient algorithms are very suitable for training of feed-forward networks. Use of second order information by calculations on the Hessian matrix can be used to improve convergence. I would like to thank Brian Mayoh for being a very inspiring advisor. Also many thanks to Ole sterby who has been a great help on many technical issues. During my visit to Carnegie Mellon University, Pittsburgh, I received inspiration and advice from many people, which I can not all thank here. A great thanks, however, to my advisor at CMU, Scott Fahlman. Thank you to the Royal Danish Research Council and the Carlsberg Foundation, who made this research possible by providing nancial support.

View PDFchevron_right
An On-line Processing and Training (OPT) Architecture for Multilayer Feedforward Neural Networks
Michael Cooke

Artificial neural networks (ANNs) are presently being utilized to solve many scientific and engineering problems. The advantages include but are not limited to increased stability and processing speeds over classical approximation methods, simplicity of development and hardware implementability. The range of problems which may benefit from the use of ANNs seems unbounded, due to the various architectures and paradigms that may be employed. ANNs fall into two primary categories: 1) supervised networks that require output targets for training; and 2) unsupervised networks that 'learn' or stabilize to each input event. The supervised networks, which include the well known backpropagation (BPN) networks, are by far the most robust. Although the supervised networks can be applied to a greater number of problems they do not respond well in situations where the operational input-output relationships may vary from the previously learned. This is due to the necessity of off-line training. This shortfall usually regulates these powerful tools to off-line processing tasks. The objective of this research effort is to develop, design and implement an on-line processing and training (OPT) supervised ANN.

View PDFchevron_right
Exact Solution for On-Line Learning in Multilayer Neural Networks
Sara Solla

Physical Review Letters, 1995

We present an analytic solution to the problem of on-line gradient-descent learning for two-layer neural networks with an arbitrary number of hidden units in both teacher and student networks. The technique, demonstrated here for the case of adaptive input-to-hidden weights, becomes exact as the dimensionality of the input space increases. Layered neural networks are of interest for their ability to implement inputoutput maps 1]. Classi cation and regression tasks formulated as a map from an N-dimensional input space onto a scalar are realized through a map = f J ( ), which can be modi ed through changes in the internal parameters fJg specifying the strength of the interneuron couplings. Learning refers to the modi cation of these couplings so as to bring the map f J implemented by the network as close as possible to a desired map f. Information about the desired map is provided through independent examples ( ; ), with = f( ) for all .

View PDFchevron_right

Related topics

  • Computer Science
  • Artificial Intelligence
  • Selective Attention
  • Machine Learning
  • Stochastic Gradient Descent
  • Activation Functions
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved