Learning Automata in Control Planning Strategies

2011

This thesis is mainly concerned with the efficient solution of a linear discrete-time finite horizon optimal control problem (FHOCP) with quadratic cost and linear constraints on the states and inputs. In predictive control, such a FHOCP needs to be solved online at each sampling instant. In order to solve such a FHOCP, it is necessary to solve a quadratic programming (QP) problem. Interior point methods (IPMs) have proven to be an efficient way of solving quadratic programming problems. A linear system of equations needs to be solved in each iteration of an IPM. The ill-conditioning of this linear system in the later iterations of the IPM prevents the use of an iterative method in solving the linear system due to a very slow rate of convergence; in some cases the solution never reaches the desired accuracy. A new well-conditioned IPM, which increases the rate of convergence of the iterative method is proposed. The computational advantage is obtained by the use of an inexact Newton ...

IFAC-PapersOnLine, 2016

Automatic control plays important role in both industrial and commercial engineering applications. Many of these applications use predictive control strategies implemented on digital computers to control dynamic systems. This paper aims to present methods used to design predictive control with the knowledge of the controlled system's model.

Automatica

This work presents DMPC (Data-and Model-Driven Predictive Control) to solve control problems in which some of the constraints or parts of the objective function are known, while others are entirely unknown to the controller. It is assumed that there is an exogenous "black box" system, e.g. a machine learning technique, that predicts the value of the unknown functions for a given trajectory. DMPC (1) provides an approach to merge both the model-based and black-box systems; (2) can cope with very little data and is sample efficient, building its solutions based on recently generated trajectories; and (3) improves its cost in each iteration until converging to an optimal trajectory, typically needing only a few trials even for nonlinear dynamics and objectives. Theoretical analysis of the algorithm is presented, proving that the quality of the trajectory does not worsen with each new iteration, as well as providing bounds on the complexity. We apply the DMPC algorithm to the motion planning of an autonomous vehicle with nonlinear dynamics.

IFAC Proceedings Volumes, 2000

An overview is first provided of some of the most common formulations of Predictive Control, such as Dynamic Matrix Control (DMC), Predictive Functional Control (PFC), Preview Control and Generalized Predictive Control (GPC). The main features, advantages and disadvantages are discussed, A new algorithm, based on a so called Linear Quadratic Generalized Predictive Controller (LQGPC) is then introduced. This algorithm combines features of Predictive Control with those of Linear Quadratic Gaussian Control, providing improved stability and robustness properties. Attention then turns to the state-space formulation of the problem. An algorithm called Dynamic Performance Predictive Control is briefly outlined and some links to the Linear Quadratic Generalized Predictive Controller are exploited. Finally, applications of new Predictive Control Techniques are discussed based on industrial examples from the Power and Steel industries. These are industrial fields where Predictive Control has not yet established a leading position. However, the new algorithms, due to their improved stability and robustness properties, have the potential to provide real quality and performance benefits.

2010 IEEE International Conference on Control Applications, 2010

In this paper a novel method called Sampling-Based Model Predictive Control (SBMPC) is proposed as an efficient MPC algorithm to generate control inputs and system trajectories. The algorithm combines the benefits of sampling-based motion planning with MPC while avoiding some of the major pitfalls facing both traditional samplingbased planning algorithms and traditional MPC. The method is based on sampling (i.e., discretizing) the input space at each sample period and implementing a goal-directed optimization method (e.g., A ) in place of standard numerical optimization. This formulation of MPC readily applies to systems with nonlinear dynamics and avoids the local minima which can limit the performance of MPC algorithms implemented using traditional, derivative-based, nonlinear programming. The SBMPC algorithm is compared with a more standard online MPC algorithm using cluttered environment navigation for an Ackerman steered vehicle and a set point problem for a nonlinear, continuous stirred-tank reactor (CSTR).

2005

Abstract: We propose the use of Model Predictive Control (MPC) for controlling systems described by Markov decision processes. First, we consider a straightforward MPC algorithm for Markov decision processes. Then, we propose value functions, a means to deal with issues arising in conventional MPC, eg, computational requirements and sub-optimality of actions. We use reinforcement learning to let an MPC agent learn a value function incrementally.

2005

In recent times, a number of machine-learning and pattern-recognition methods have been used to solve operations-management problems in the real world. In this report, we focus on a powerful learning automata algorithm, from the field of pattern-recognition and machine vision, to solve stochastic discrete-optimization problems encountered in the industry. The algorithm is due to Thathachar and Sastry . We present an empirical study of this technique on two real-world test-beds of interest in engineering management. The first test-bed, which is from production management, is a complex buffer-optimization problem that defies exact theoretical analysis. The other test-bed is related to revenue management from the airline domain where also it is difficult to develop an exact theoretical model. We find that the learning automata technique produces encouraging computational results. We also discuss some analytical properties of the algorithm.

Journal of Process Control, 1998

In industrial practice, constrained steady state optimisation and predictive control are separate, albeit closely related functions within the control hierarchy. This paper presents a method which integrates predictive control with on-line optimisation with economic objectives. A receding horizon optimal control problem is formulated using linear state space models. This optimal control problem is very similar to the one presented in many predictive control formulations, but the main difference is that it includes in its formulation a general steady state objective depending on the magnitudes of manipulated and measured output variables. This steady state objective may include the standard quadratic regulatory objective, together with economic objectives which are often linear. Assuming that the system settles to a steady state operating point under receding horizon control, conditions are given for the satisfaction of the necessary optimality conditions of the steady-state optimisation problem. The method is based on adaptive linear state space models, which are obtained by using on-line identification techniques. The use of model adaptation is justified from a theoretical standpoint and its beneficial effects are shown in simulations. The method is tested with simulations of an industrial distillation column and a system of chemical reactors.

Studies in Systems, Decision and Control, 2019

Model predictive control has been the subject of intensive research for the last three decades [8, 25, 47]. This culminated in many practical applications [1, 7, 20]. The attractiveness of predictive control algorithms comes from their ability to consider process and technological constraints imposed on input, output or state variables. Another very important reason is that the operating principles are understandable and relatively easy to explain to practitioners, which seems to be a crucial aspect when it comes to an industrial implementation of a new control scheme. This explains the widespread use of predictive control algorithms in the petrochemical and related industries, where the fact that the imposed constraints are made sure to be satisfied is particularly important. Predictive control is a modern control strategy that derives the control sequence by solving at each sampling time a finite-horizon open-loop optimal control problem. The important role is played by a model of the process used to predict future plant outputs based on the past and current outputs as well as on future control signals. These are calculated by means of the minimization of the cost function taking into account constraints. In many cases, the constraints are in the form of inequalities imposed on the process variables, e.g. the input saturations. If such nonlinearities are not taken into account, it can result in a degraded performance of closed-loop control and lead to stability problems. As pointed out in the outstanding survey papers of Morari and Lee [27] and Mayne and colleagues [26], the predictive control theory has reached a high level of maturity, especially in the case of linear systems. However, nonlinear systems remain problematic, for a number of reasons. The most common issues arise in relation to the following areas: the modelling of nonlinear processes, state estimation, robustness or fault-tolerant control [27]. The ideas described in this chapter can be viewed as arising as a response to the emerging problems encountered in predictive control. In order to deal with nonlinearities, artificial neural networks are used. The neural model can be used as a one-step ahead predictor, which is then run recursively to obtain a k-step ahead

Engineering Applications of Artificial Intelligence, 2016

Systems with a priori unknown and time-varying dynamic behavior pose a significant challenge in the field of Nonlinear Model Predictive Control (NMPC). When both the identification of the nonlinear system and the optimization of control inputs are done robustly and efficiently, NMPC may be applied to control such systems. This paper considers stable systems and presents a novel method for adaptive NMPC, called Adaptive Sampling Based Model Predictive Control (Adaptive SBMPC), that combines a radial basis function neural network identification algorithm with a nonlinear optimization method based on graph search. Unlike other NMPC methods, it does not rely on linearizing the system or gradient based optimization. Instead, it discretizes the input space to the model via pseudo-random sampling and feeds the sampled inputs through the nonlinear model, producing a searchable graph. For this discretization, an optimal path is found using Lifelong Planning A * , an efficient graph search method. Adaptive SBMPC is used in simulation to identify and control a simple plant with clearly visualized nonlinear behavior. In these simulations, both fixed and time-varying dynamic systems are considered. Results are compared with an adaptive version of Neural GPC, an existing NMPC algorithm based on Newton-Raphson optimization and a back propagation neural network model. When the cost function exhibits many local minima, Adaptive SBMPC is successful in finding a low-cost solution that appears close globally optimal while Neural GPC converges to a solution that is only locally optimal.