Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
References
All Topics
Computer Science
Machine Learning

Some topics in neural networks and control

Profile image of Eduardo D. SontagEduardo D. Sontag

1993

Abstract

This report constitutes an expanded version of a presentation given by the author at the 1993 European Control Conference (short course on "Neural Nets for Control"). The first part places neurocontrol techniques in a general learning control framework. The second part of the report, which is essentially independent of the first, briefly surveys several basic theoretical results regarding neural networks.

Related papers

Some Control Theoretic Issues in Neural Networks
Chaouki Abdallah

1970

We have observed that many neural network models can be written as a bilinear system with a speci c form of nonlinear state-to-input feedback. This framework includes the ART architecture among others. There are two signi cant results which follow from this observation. First, the parameters of the model determine the controllability of the system. A system is controllable if there exists some input which transfers any initial state to any desired nal state in a nite time. If for a given set of these parameters the system is not controllable, then there are regions of the state space which the system can never enter in a nite time for any input. Because of this restriction the learning ability of the system may be severely limited. Second, the multiplicative equation is linear in all of the parameters, and all of the adjustable weights. This means that a provably convergent learning algorithm can be devised for all of these quantities. This does not however circumvent the learning limitation since the learning algorithm is not guaranteed to converge in a nite time. In the paper, we will study these issues as they apply to the ART architecture.

View PDFchevron_right
An introduction to the use of neural networks in control systems
Orlando De Jesus

International Journal of Robust and Nonlinear Control, 2002

The purpose of this paper is to provide a quick overview of neural networks and to explain how they can be used in control systems. We introduce the multilayer perceptron neural network and describe how it can be used for function approximation. The backpropagation algorithm (including its variations) is the principal procedure for training multilayer perceptrons; it is briefly described here. Care must be taken, when training perceptron networks, to ensure that they do not overfit the training data and then fail to generalize well in new situations. Several techniques for improving generalization are discussed. The paper also presents three control architectures: model reference adaptive control, model predictive control, and feedback linearization control. These controllers demonstrate the variety of ways in which multilayer perceptron neural networks can be used as basic building blocks. We demonstrate the practical implementation of these controllers on three applications: a con...

View PDFchevron_right
Artificial neural networks in control and optimization
Cairo L. Nascimento Jr.

Doctor Thesis. University of Manchester. …, 1994

View PDFchevron_right
Neural networks as a control methodology
Claire McCullough

1990

View PDFchevron_right
An Overview of Neural Networks Results for Systems and Control
Chaouki Abdallah
View PDFchevron_right
Artificial Neural Network Applications to Control
Prof. Dr. Şahin YILDIRIM

The use of Artificial Neural Networks (ANNs) for the control of robots has recently become an active research topic in control engineering. This paper is presented a review of current research in control of robotic systems based on ANN techniques. The main concepts of ANNs are presented and the mathematical background of supervised learning in multilayer networks is outlined in first section. A brief summary on adaptive control of dynamic systems is given second section. The application of ANNs to control systems is discussed in the last section. uygulanması detaylıca sunulmuştur. ANNs are made up of simple, highly interconnected processing units called neurons each of which performs two functions: aggregation of its inputs from other neurons or the external environment and generation of an output from the aggregated inputs. The output from a neuron is fed to other neurons to which it is connected via weighted links. Through this simple structure, ANNs have been shown to be able to ...

View PDFchevron_right
Neural Network Control Systems that Learn to Perform Appropriately
Patricia Riddell

International Journal of Neural Systems, 2001

Setting up a neural network with a learning algorithm that determines how it can best operate is an efficient way to formulate control systems for many engineering applications, and is often much more feasible than direct programming. This paper examines three important aspects of this approach: the details of the cost function that is used with the gradient descent learning algorithm, how the resulting system depends on the initial pre-learning connection weights, and how the resulting system depends on the pattern of learning rates chosen for the different components of the system. We explore these issues by explicit simulations of a toy model that is a simplified abstraction of part of the human oculomotor control system. This allows us to compare our system with that produced by human evolution and development. We can then go on to consider how we might improve on the human system and apply what we have learnt to control systems that have no human analogue.

View PDFchevron_right
Neural Control for Nonlinear Dynamic Systems
Ssu-Hsin Yu

A neural network based approach is presented for controlling two distinct types of nonlinear systems. The first corresponds to nonlinear systems with parametric uncertainties where the parameters occur nonlinearly. The second corresponds to systems for which stabilizing control structures cannot be determined. The proposed neural controllers are shown to result in closed-loop system stability under certain conditions.

View PDFchevron_right
Neural network learning and learning control through optimization techniques [microform] /
Homayoon Beigi
View PDFchevron_right
Neural Networks for Robot Control
Gabriela Cembrano

Artificial Intelligence in Real-Time Control 1994, 1995

This peper describes the use of neural networks in diferent domains of robot control. Three robot control problems, relevant to a broad range of robotics applications, are analyzed, with a review of the state of the art and a description of current research by the authors, highlighting the advantages of the use of neural networks with respect to conventional techniques.

View PDFchevron_right

References (66)

  1. Albertini, F., and E.D. Sontag, "For neural networks, function determines form," Neural Networks, 1993, to appear. See also Proc. IEEE Conf. Decision and Control, Tucson, Dec. 1992, IEEE Publications, 1992, pp. 26-31.
  2. Albertini, F., E.D. Sontag, and V. Maillot, "Uniqueness of weights for neural net- works," in Artificial Neural Networks with Applications in Speech and Vision (R. Mammone, ed.), Chapman and Hall, London, 1993.
  3. Anthony, M., and N.L. Biggs, Computational Learning Theory: An Introduction, Cambridge U. Press, 1992.
  4. Aubin, J.-P., Mathematical Methods of Artificial Intelligence, Cambridge University Press, 1993, to appear. NEURAL NETWORKS AND CONTROL 32
  5. Baker, W.L., and J.A. Farrell, "An introduction to connectionist learning control systems," in [64].
  6. Balcázar, J.L., R. Gavaldà, H. Siegelmann, and E.D. Sontag, "Some structural com- plexity aspects of neural computation," in Proc. 8th Annual IEEE Conf. Structure in Complexity Theory, San Diego, May 1992.
  7. Barron, A.R., "Neural net approximation," in Proc. Seventh Yale Workshop on Adap- tive and Learning Systems, Yale University, 1992, pp. 69-72.
  8. Barto, A.G., "Connectionist learning for control: An overview," in [40].
  9. Benedek, G.M., and A. Itai, "Learnability by fixed distributions," Theoretical Comp. Sci. 86(1991): 377-389.
  10. Baum, E.B., and D. Haussler, "What size net gives valid generalization?," Neural Computation 1(1989): 151-160.
  11. Blum, A., and R.L. Rivest, "Training a 3-node neural network is NP-complete," in Advances in Neural Information Processing Systems 2 (D.S. Touretzky, ed), Morgan Kaufmann, San Mateo, CA, 1990, pp. 9-18.
  12. Blumer, A., A. Ehrenfeucht, D. Haussler, and M. Warmuth, "Classifying learnable geometric concepts with the Vapnik-Chervonenkis dimension," in Proc. 18th. Annual ACM Symposium on Theory of Computing, pp. 273-282, ACM, Salem, 1986.
  13. Boyd, S., and L.O. Chua, "Uniqueness of a basic nonlinear structure" IEEE Trans. Circuits and Systems CAS-30(1983): 648-651.
  14. Carroll, S.M., and B.W. Dickinson, B.W., "Construction of neural nets using the Radon transform," in Proc. 1989 Int. Joint Conf. Neural Networks, pp. I: 607-611.
  15. Chen, F.C., and H.K. Khalil, "Adaptive control of nonlinear systems using neural networks," Proc. IEEE Conf. Decision and Control, Hawaii, Dec. 1990, IEEE Publi- cations, 1990.
  16. Cybenko, G., "Approximation by superpositions of a sigmoidal function," Math. Control, Signals, and Systems 2(1989): 303-314.
  17. Darken, C., M. Donahue, L. Gurvits, and E. Sontag, "Rate of approximation results motivated by robust neural network learning," in Proc. Sixth ACM Workshop on Computational Learning Theory, Santa Cruz, July 1993.
  18. Fefferman, C., "Reconstructing a neural net from its output," submitted (preprint, Mathematics Dept., Princeton, 1993).
  19. Flick, T.E., L.K. Jones, R.G. Priest, and C. Herman, "Pattern classification using projection pursuit," Pattern Recognition 23(1990): 1367-1376.
  20. Funahashi, K., "On the approximate realization of continuous mappings by neural networks," Neural Networks 2(1989): 183-192.
  21. Franklin, J.A., "Historical perspective and state of the art in connectionist learning control," Proc. IEEE Conf. Decision and Control, Tampa, Dec. 1989, IEEE Publica- tions, 1989.
  22. Grossberg, S., "Some Nonlinear Networks Capable of Learning a Spatial Pattern of Arbitrary Complexity," Proceedings of the National Academy of Sciences, USA 59(1968): 368-372.
  23. Haussler, D., "Decision theoretic generalizations of the PAC model for neural net and other learning applications," Information and Computation 100(1992): 78-150.
  24. Hebb, D.O., The Organization of Behavior, Wiley, New York, 1949.
  25. Hertz, J., A. Krogh, and R.G. Palmer, Introduction to the Theory of Neural Computa- tion, Addison-Wesley, Redwood City, 1991.
  26. Hopfield, J.J., "Neurons with graded responses have collective computational prop- erties like those of two-state neurons," Proc. of the Natl. Acad. of Sciences, USA 81(1984): 3088-3092.
  27. Hornik, K., "Approximation capabilities of multilayer feedforward networks," Neural Networks 4(1991): 251-257.
  28. NEURAL NETWORKS AND CONTROL 33
  29. Hunt, K.J., D. Sbarbaro, R. Zbikowski, and P.J. Gawthrop, "Neural networks for control systems: A survey," Automatica 28(1992): 1083-1122.
  30. Judd, J.S., Neural Network Design and the Complexity of Learning, MIT Press, Cambridge, MA, 1990.
  31. Killian, J., and H.T. Siegelmann, "On the power of sigmoid neural networks," in Proc. Sixth ACM Workshop on Computational Learning Theory, Santa Cruz, July 1993.
  32. Kohonen, T., Self-Organization and Associative Memory (3rd ed.), Springer-Verlag, Berlin, 1989.
  33. Kulkarni, S.R. Problems of Computational and Information Complexity in Machine Vision and Learning, Ph.D. Thesis, Department of Electrical Engineering and Com- puter Science, M.I.T., 1991.
  34. Lane, S.H., D. Handelman, J. Gelfand, and M. Flax, "Function approximation using multi-layered neural networks with B-spline receptive fields," Advances in Neural Information Processing Systems 3, (R.P. Lippmann, J. Moody, D.S. Touretzky, eds.), Morgan Kaufmann (San Mateo, 1991).
  35. Leshno, M., V.Ya. Lin, A. Pinkus, and S. Schocken, "Multilayer feedforward networks with a non-polynomial activation function can approximate any function," Neural Networks, 1993.
  36. Livstone, M.M., J.A. Farrell, and W.L. Baker, "A computationally efficient algorithm for training recurrent connectionist networks," in Proc. Amer. Automatic Control Conference, Chicago, June 1992.
  37. Macintyre, M., and E.D. Sontag, "Finiteness results for sigmoidal 'neural' networks," in Proc. 25th Annual Symp. Theory Computing, San Diego, May 1993.
  38. Matthews, M., "A state-space approach to adaptive nonlinear filtering using recurrent neural networks," Proc. 1990 IASTED Symp. on Artificial Intelligence Applications and Neural Networks, Zürich, pp. 197-200, July 1990.
  39. McBride, L.E., and K.S. Narendra, "Optimization of time-varying systems," IEEE Trans. Autom. Control, 10(1965): 289-294.
  40. McCulloch, W.S., and W. Pitts, "A logical calculus of the ideas immanent in nervous activity," Bull. Math. Biophys. 5(1943): 115-133.
  41. Miller, T., R.S. Sutton, and P.J. Werbos (eds.), Neural networks For Control, MIT Press, Cambridge, 1990.
  42. Minsky, M., and S. Papert, Perceptrons: An Introduction to Computational Geometry, Expanded Edition, MIT Press, Cambridge, 1988.
  43. Niranjan, M. and F. Fallside, "Neural networks and radial basis functions in classifying static speech patterns," Computer Speech and Language 4 (1990): 275-289.
  44. Pisier, G., "Remarques sur un resultat non publiè de B. Maurey," in Seminaire d'analyse fonctionelle 1980-1981, Ecole Polytechnique, Palaiseau, 1981.
  45. Poggio, T., and F. Girosi, "Networks for approximation and learning," Proc. of the IEEE 78(1990): 1481-1497.
  46. Polycarpou, M.M., and P.A. Ioannou, "Neural networks and on-line approximators for adaptive control," in Proc. Seventh Yale Workshop on Adaptive and Learning Systems, pp. 93-798, Yale University, 1992.
  47. Rosenblatt, F., Principles of Neurodynamics, Spartan Books, New York, 1962.
  48. Rumelhart, D.E., and J.L. McClelland, Parallel Distributed Processing, vol. 1, MIT Press, Cambridge, 1986.
  49. Scalero, R.S., and N. Tepedelenlioglu, "Fast Algorithm for Training Feed-forward Neural Networks," IEEE Trans. Signal Processing TSP-40(1992): 202-210.
  50. Schiffmann, W., M. Joost, and R. Werner, "Optimization of the Backpropagation Algorithm for Training Multilayer Perceptrons," Tech Report, University of Koblenz, Institute of Physics, January 1993. NEURAL NETWORKS AND CONTROL 34
  51. Schmidhuber, J., "Accelerated Learning in Back-Propagation Nets," in Connectionism in Perspective, Elsevier, 1989, pp. 439-445.
  52. Siegelmann, H.T., and E.D. Sontag, "On the computational power of neural nets," in Proc. Fifth ACM Workshop on Computational Learning Theory, Pittsburgh, July 1992, 440-449. Extended version to appear in J. Computer Syst. Sci..
  53. Siegelmann, H.T., and E.D. Sontag, "Some results on computing with 'neural nets'," Proc. IEEE Conf. Decision and Control, Tucson, Dec. 1992, IEEE Publications, 1992, pp. 1476-1481.
  54. Siegelmann, H.T., and E.D. Sontag, "Analog computation, neural networks, and circuits," Theor. Comp. Sci., to appear.
  55. Slotine,J.-J., and R.M. Sanner, "Neural networks for adaptive control and recursive identification: a theoretical framework," in Essays on Control: Perspectives in the Theory and its Applications (H.L. Trentelman and J.C. Willems, eds.), Birkhauser, Boston, 1993, pp. 381-433.
  56. Smith, W.W., and W.J. Rugh, "On the structure of a class of nonlinear systems," IEEE Trans. Automatic Control AC-19(1974): 701-706.
  57. Sontag, E.D., Mathematical Control Theory: Deterministic Finite Dimensional Sys- tems, Springer, New York, 1990.
  58. Sontag, E.D., "Feedforward nets for interpolation and classification," J. Comp. Syst. Sci. 45(1992): 20-48.
  59. Sontag, E.D., "Feedback stabilization using two-hidden-layer nets," IEEE Trans. Neural Networks 3 (1992): 981-990.
  60. Sontag, E.D., and H.J. Sussmann, "Backpropagation separates where perceptrons do," Neural Networks, 4(1991): 243-249.
  61. Sprecher, D.A. "On the structure of continuous functions of several variables," Trans- actions of the American Mathematical Society 115(1965): 340-355.
  62. Stinchcombe, M., and White, H. "Approximating and learning unknown mappings using multilayer feedforward networks with bounded weights," in Proc. 1990 Inter- national Joint Conf. Neural Networks, pp. III: 7-16, IEEE Press, New York, 1990.
  63. Sussmann, H.J., "Uniqueness of the weights for minimal feedforward nets with a given input-output map," Neural Networks 5(1992): 589-593.
  64. Vapnik, V.N., Estimation of Dependencies Based on Empirical Data, Springer, Berlin, 1982.
  65. White, D.A., and D.A. Sofge (eds.), Handbook of Intelligent Control: Neural, Fuzzy and Adaptive Approaches, Van Nostrand Reinhold, NY, 1992.
  66. Weiss, S.M., and C.A. Kulikowski,Computer Systems That Learn, Morgan Kaufmann, San Mateo, CA, 1991. NEURAL NETWORKS AND CONTROL 35

Related papers

NEURAL NETWORKS FOR CONTROL
Eduardo D. Sontag

1993

This paper starts by placing neural net techniques in a general nonlinear control framework. After that, several basic theoretical results on networks are surveyed.

View PDFchevron_right
Neural networks for control: Edited by T. Miller, RS Sutton, and PJ Werbos, MIT Press, Cambridge, MA: 1990, hardcover $49.95, 524 pp., ISBN 0-262-13261-3
Daniel Bullock

Neural Networks for Control is based on a workshop entitled "The Application of Neural Networks to Robotics and Control" held in October 1988 ~ at the University of New Hampshire. Consistent with the varied backgrounds of the workshop attendees, this collection of works endeavors not only to present current theoretical research in neural network-based control systems, but also aims to identify industrial applications where neural networks may prove useful in the near future. The book is broken into three parts: (I) general principles, (II) motion control, and (III) application domains. In addition, eight benchmark control problems are included in an appendix. The main theme pervading this effort concerns the synthesis of neural network methods with existing knowledge from the well-studied fields of conventional and adaptive control theory. Perhaps due to the focus on engineering control theory, little direct contact is made with issues involving control in biological systems. Control problems arise when we try to make a physical system exhibit a desired response. If we know the functions relating inputs to outputs and vice versa for the system, we can use these functions to predict system behavior and control the system's response by judicious choice of inputs. Unfortunately, most complex systems exhibit highly nonlinear input-output functions. Moreover, because the complexity of such systems is often necessary for their intended function (e.g., controlling the position and orientation of a hand in 3-D space requires at least a 6 degree-of-freedom "arm") the input-output function of the system cannot be simplified. Therefore, it becomes necessary to solve a difficult prediction problem as a preliminary to effective control. A classical approach to predicting the behavior of such systems is to "open up the black box" (i.e., to build a model of the system that captures its constituent structure and the elementary functions describing interactions among the constituents). An accurate model allows the overall input-output function to be computed as a composition of more elementary functions. Moreover, to paraphrase the Swiss structuralist Piaget, "to understand is to reconstruct." The description of the constituent structure taken together with the local laws of interaction provide an understanding of the mechanisms by which the aggregate input-output function is generated. However, another path can be taken to predicting the response of a system exhibiting a highly nonlinear input-output function: perform a variety of nonlinear regression analysis, or least-squares curve fitting, on a sufficiently broad sample of input-output pairs. Such an analysis can often yield suffi-Despite the 1988 date of the workshop, references as recent as 1990 can he found in the book.

View PDFchevron_right
Neural control theory: an overview
Johan Suykens

1996

In this paper we present a short introduction to the theory of neural control. Universal approximation, on-and o -line learning ability and parallelism of neural networks are the main motivation for their application to modelling and control problems. This has led to several existing neural control strategies. An overview of methods is presented, with emphasis on the foundations of neural optimal control, stability theory and nonlinear system identi cation using neural networks for modelbased neural control design.

View PDFchevron_right
Neural networks for control systems
Panos Antsaklis

IEEE Transactions on Neural Networks, 1990

View PDFchevron_right
Introduction to neural networks for intelligent control
Behnam Bavarian

Control Systems Magazine, IEEE, 1988

Introduction The purpose of this paper is to serve as an introduction to neural networks and to sug-gest that neural network architecture can be used for design and application of intelligent control systems [ 11. The neural network ap-proach is introduced by defining the problem ...

View PDFchevron_right

Related topics

  • Neural Net
  • Neural network control
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved