Quasi-Newton Methods

New approximate secant equations are shown to result from the knowledge of (problem dependent) invariant subspace information, which in turn suggests improvements in quasi-Newton methods for unconstrained minimization. It is also shown that this type of information may often be extracted from the multigrid structure of discretized infinite dimensional problems. A new limited-memory BFGS using approximate secant equations is then derived and its encouraging behaviour illustrated on a small collection of multilevel optimization examples. The smoothing properties of this algorithm are considered next, and automatic generation of approximate eigenvalue information demonstrated. The use of this information for improving algorithmic performance is finally investigated on the same multilevel examples.

New approximate secant equations are shown to result from the knowledge of (problem dependent) invariant subspace information, which in turn suggests improvements in quasi-Newton methods for unconstrained minimization. A new limitedmemory BFGS using approximate secant equations is then derived and its encouraging behaviour illustrated on a small collection of multilevel optimization examples. The smoothing properties of this algorithm are considered next, and automatic generation of approximate eigenvalue information demonstrated. The use of this information for improving algorithmic performance is finally investigated on the same multilevel examples.

The properties of multilevel optimization problems defined on a hierarchy of discretization grids can be used to define approximate secant equations, which describe the second-order behaviour of the objective function. Following earlier work by Gratton and Toint (2009), we introduce a quasi-Newton method (with a linesearch) and a nonlinear conjugate gradient method that both take advantage of this new second-order information. We then present numerical experiments with these methods and formulate recommendations for their practical use. Keywords nonlinear optimization • multilevel problems • quasi-Newton methods • nonlinear conjugate gradient methods • limited-memory algorithms.

In this paper we analyze the use of structured quasi-Newton formulae as preconditioners of iterative linear methods when the inexact-Newton approach is employed for solving nonlinear systems of equations. We prove that superlinear convergence and bounded work per iteration is obtained if the preconditioners satisfy a Dennis-Moré condition. We develop a theory of Least-Change Secant Update preconditioners and we present an application concerning a structured BFGS preconditioner.

A random search algorithm for unconstrained local nonsmooth optimization is described. The algorithm forms a partition on R n using classification and regression trees (CART) from statistical pattern recognition. The CART partition defines desirable subsets where the objective function f is relatively low, based on previous sampling, from which further samples are drawn directly. Alternating between partition and sampling phases provides an effective method for nonsmooth optimization. The sequence of iterates {z k } is shown to converge to an essential local minimizer of f with probability one under mild conditions. Numerical results are presented to show that the method is effective and competitive in practice.

The aim of this work is to test the Levemberg Marquardt and BFGS (Broyden Fletcher Goldfarb Shanno) algorithms, implemented by the matlab functions lsqnonlin and fminunc of the Optimization Toolbox, for modeling the kinetic terms occurring in chemical processes of adsorption. We are interested in tests with noisy data that are obtained by adding Gaussian random noise to the solution of a model with known parameters. While both methods are very precise with noiseless data, by adding noise the quality of the results is greatly worsened. The semiconvergent behaviour of the relative error curves is observed for both methods. Therefore a stopping criterion, based on the Discrepancy Principle is proposed and tested. Great improvement is obtained for both methods, making it possible to compute stable solutions also for noisy data.

At each iteration of the augmented Lagrangian algorithm, a nonlinear subproblem is being solved. The number of inner iterations (of some/any method) needed to obtain a solution of the subproblem, or even a suitable approximate stationary point, is in principle unknown. In this paper we show that to compute an approximate stationary point sufficient to guarantee local superlinear convergence of the augmented Lagrangian iterations, it is enough to solve two quadratic programming problems (or two linear systems in the equality-constrained case). In other words, two inner Newtonian iterations are sufficient. To the best of our knowledge, such results are not available even under the strongest assumptions (of second-order sufficiency, strict complementarity, and the linear independence constraint qualification). Our analysis is performed under second-order sufficiency only, which is the weakest assumption for obtaining local convergence and rate of convergence of outer iterations of the augmented Lagrangian algorithm. The structure of the quadratic problems in question is related to the stabilized sequential quadratic programming and to second-order corrections.

The stabilized version of the sequential quadratic programming algorithm (sSQP) had been developed in order to achieve fast convergence despite possible degeneracy of constraints of optimization problems, when the Lagrange multipliers associated to a solution are not unique. Superlinear convergence of sSQP had been previously established under the strong second-order sufficient condition for optimality (without any constraint qualification assumptions). We prove a stronger superlinear convergence result than the above, assuming the usual second-order sufficient condition only. In addition, our analysis is carried out in the more general setting of variational problems, for which we introduce a natural extension of sSQP techniques. In the process, we also obtain a new error bound for Karush-Kuhn-Tucker systems for variational problems that holds under an appropriate second-order condition.

A set of four-dimensional variational data assimilation (4D-Var) experiments were conducted using both a standard method and an incremental method in an identical twin framework. The full physics adjoint model of the Florida State University global spectral model (FSUGSM) was used in the standard 4D-Var, while the adjoint of only a few selected physical parameterizations was used in the incremental method. The impact of physical processes on 4D-Var was examined in detail by comparing the results of these experiments. The inclusion of full physics turned out to be significantly beneficial in terms of assimilation error to the lower troposphere during the entire minimization process. The beneficial impact was found to be primarily related to boundary layer physics. The precipitation physics in the adjoint model also tended to have a beneficial impact after an intermediate number (50) of minimization iterations. Experiment results confirmed that the forecast from assimilation analyses with the full physics adjoint model displays a shorter precipitation spinup period. The beneficial impact on precipitation spinup did not result solely from the inclusion of the precipitation physics in the adjoint model, but rather from the combined impact of several physical processes. The inclusion of full physics in the adjoint model exhibited a detrimental impact on the rate of convergence at an early stage of the minimization process, but did not affect the final convergence. A truncated Newton-like incremental approach was introduced for examining the possibility of circumventing the detrimental aspects using the full physics in the adjoint model in 4D-Var but taking into account its positive aspects. This algorithm was based on the idea of the truncated Newton minimization method and the sequential cost function incremental method introduced by Courtier et al., consisting of an inner loop and an outer loop. The inner loop comprised the incremental method, while the outer loop consisted of the standard 4D-Var method using the full physics adjoint. The limited-memory quasi-Newton minimization method (L-BFGS) was used for both inner and outer loops, while information on the Hessian of the cost function was jointly updated at every iteration in both loops. In an experiment with a two-cycle truncated Newton-like incremental approach, the assimilation analyses turned out to be better than those obtained from either the standard 4D-Var or the incremental 4D-Var in all aspects examined. The CPU time required by this two-cycle approach was larger by 35% compared with that required by the incremental 4D-Var without almost any physics in the adjoint model, while the CPU time required by the standard 4D-Var with the full physics adjoint model was more than twice that required by the incremental 4D-Var. Finally, several hypotheses concerning the impact of using standard 4D-Var full physics on minimization convergence were advanced and discussed.

In this paper, the performances of the quasi-Newton BFGS algorithm, the NEWUOA derivative free optimizer, the Covariance Matrix Adaptation Evolution Strategy (CMA-ES), the Differential Evolution (DE) algorithm and Particle Swarm Optimizers (PSO) are compared experimentally on benchmark functions reflecting important challenges encountered in real-world optimization problems. Dependence of the performances in the conditioning of the problem and rotational invariance of the algorithms are in particular investigated.

A neural network (NN)-based nonlinear predictive control (NPC) is described for control of turbine power with variation in gate position. The studied plant includes the tunnel, surge tank and penstock effect dynamics. Multilayer perceptron neural network is chosen to represent a neural network nonlinear autoregressive with exogenous signal model of hydro power plant. With the said NN model configuration, quasi-Newton and Levenberg-Marquardt iterative optimization algorithms are applied in order to determine optimal predictive control parameters. The controlled response is simulated on different amplitude step function and trapezoidal shape reference signal. The study also discusses comparison with an approximate predictive control approach, being linearized around operating points. It is shown that NPC strategy gives impressive results in comparison to the approximated one.

In the Fourier-domain flattening methods presented previously (Lomask and Claerbout, 2002; Lomask, 2003; Lomask et al., 2005) the data has to be mirrored in order to eliminate Fourier artifacts. This means that the data is replicated and reversed so that the boundaries are periodic. This requires four times the memory in 2D and eight times in 3D. In a world where poststack

This article presents an optimization methodology for finding the heater settings that provide spatially uniform transient heating in manufacturing processes involving radiant heating. Equations governing the transient temperature and temperature sensitivity distributions over the product are first derived using an infinitesimal-area technique and then solved numerically to calculate the objective function and gradient vector. Minimization is done using a quasi-Newton algorithm that incorporates an active set method to enforce design constraints. This methodology is demonstrated by finding the optimal transient heater settings of a two-dimensional annealing furnace.

A design procedure for frequency-response masking (FRM) prototype filters of cosine-modulated filter banks (CMFBs) is proposed. In the given method, we perform minimization of the maximum attenuation level in the filter's stopband, subject to intersymbol interference (ISI) and intercarrier interference (ICI) constraints. For optimization, a quasi-Newton algorithm with line search is used, and we provide simplified analytical expressions to impose the interference constraints, which greatly reduce the computational complexity of the optimization procedure. The result is lower levels of ISI and ICI for a predetermined filter order, or a reduced filter complexity for given levels of interferences. It is then illustrated how the FRM-CMFB structure is suitable for implementing filter banks with a large number of bands, yielding sharp transition bands and small roll-off factors, which is an attractive feature for a wide range of practical applications.

A new algorithm for the location of a transition-state structure on an energy hypersurface is proposed. The method is compared to three other quasi-Newton step calculations available in literature. Numerical results derived from several examples are compared to those obtained by the two algorithms implemented in the Gaussian package.

PREQN is a package of Fortran 77 subroutines for automatically generating preconditioners for the conjugate gradient method. It is designed for solving a sequence of linear systems A i x = b i i = 1 : : : t , where the coe cient matrices A i are symmetric and positive de nite and vary slowly. Problems of this type arise, for example, in nonlinear optimization. The preconditioners are based on limited memory quasi-Newton updating and are recommended for problems in which: i) the coe cient matrices are not explicitly known and only matrix-vector products of the form A i v can be computed or ii) the coe cient matrices are not sparse. PREQN is written so that a single call from a conjugate gradient routine performs the preconditioning operation and stores information needed for the generation of a new preconditioner. Categories and Subject Descriptors: G.1 Numerical Analysis]: G 1.3 Numerical Linear Algebra|Linear systems G 1.6 Optimization|Unconstrained optimization G.4 Mathematical Software]:|E ciency Reliability and robustness.

The quasi-Newton strategy presented in this paper preserves one of the most important features of the stabilized Sequential Quadratic Programming method, the local convergence without constraint qualifications assumptions. It is known that the primal-dual sequence converges quadratically assuming only the second-order sufficient condition. In this work, we show that if the matrices are updated by performing a minimization of a Bregman distance (which includes the classic updates), the quasi-Newton version of the method converges superlinearly without introducing further assumptions. Also, we show that even for an unbounded Lagrange multiplier set, the generated matrices satisfies a bounded deterioration property and the Dennis-Moré condition.

The stabilized version of the sequential quadratic programming algorithm (sSQP) had been developed in order to achieve fast convergence despite possible degeneracy of constraints of optimization problems, when the Lagrange multipliers associated to a solution are not unique. Superlinear convergence of sSQP had been previously established under the strong second-order sufficient condition for optimality (without any constraint qualification assumptions). We prove a stronger superlinear convergence result than the above, assuming the usual second-order sufficient condition only. In addition, our analysis is carried out in the more general setting of variational problems, for which we introduce a natural extension of sSQP techniques. In the process, we also obtain a new error bound for Karush-Kuhn-Tucker systems for variational problems that holds under an appropriate second-order condition.

In this article, we consider the correction of metric matrices in quasi-Newton methods (QNM) from the perspective of machine learning theory. Based on training information for estimating the matrix of the second derivatives of a function, we formulate a quality functional and minimize it by using gradient machine learning algorithms. We demonstrate that this approach leads us to the well-known ways of updating metric matrices used in QNM. The learning algorithm for finding metric matrices performs minimization along a system of directions, the orthogonality of which determines the convergence rate of the learning process. The degree of learning vectors' orthogonality can be increased both by choosing a QNM and by using additional orthogonalization methods. It has been shown theoretically that the orthogonality degree of learning vectors in the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method is higher than in the Davidon-Fletcher-Powell (DFP) method, which determines the advantage of the BFGS method. In our paper, we discuss some orthogonalization techniques. One of them is to include iterations with orthogonalization or an exact one-dimensional descent. As a result, it is theoretically possible to detect the cumulative effect of reducing the optimization space on quadratic functions. Another way to increase the orthogonality degree of learning vectors at the initial stages of the QNM is a special choice of initial metric matrices. Our computational experiments on problems with a high degree of conditionality have confirmed the stated theoretical assumptions.

The notion of least-change secant updates is extended to apply to nonsquare matrices in a way appropriate for quasi-Newton methods used to solve systems of nonlinear equations that depend on parameters. Extensions of the widely used least-change secant updates for square matrices are given. A local convergence analysis for certain paradigm iterations is outlined as motivation for the use of these updates, and numerical experiments involving these iterations are discussed. Key words, least-change secant updates, quasi-Newton updates, parameter-dependent systems AMS(MOS) subject classification. 65H10 1. Introduction. Quasi-Newton methods are very widely used iterative methods for solving systems of nonlinear algebraic equations. The basic form of a quasi-Newton method for solving F(x) 0, F R n-. Rn, is (1.1) Xk+: =xk-B:F(xk), in which Bk ,. F(xk) E Rnn, the Jacobian (matrix) of F at xk. For practical success, it is usually essential to augment this basic form with procedures for modifying the step-B:F(xk) to ensure progress from bad starting points, but we need not consider such procedures here. For a general reference on all aspects of quasi-Newton methods, see Dennis and Schnabel [11]. The most effective quasi-Newton methods are those in which each successive Bk+: is determined as a least-change secant update of its predecessor Bk. As the name suggests, Bk+l is determined as a least-change secant update of Bk by making the least possible change in Bk (as measured by a suitable matrix norm) which incorporates current secant information (usually expressed in terms of successive x-and F-values) and other available information about the structure of F. There are also notable updates which, strictly speaking, are least-change inverse secant updates obtained in an analogous way by making the least possible change in B-:. When speaking generically of least-change secant updates, we intend to include these. In [10], Dennis and Schnabel precisely formalize the notion of a least-change secant update and show how the most widely used updates can be derived as least-change secant updates. In [12], Dennis and Walker show that least-change secant update methods, i.e., quasi-Newton methods which use least-change secant updates, have desirable convergence properties in general.

The notion of least-change secant updates is extended to apply to nonsquare matrices in a way appropriate for quasi-Newton methods used to solve systems of nonlinear equations that depend on parameters. Extensions of the widely used least-change secant updates for square matrices are given. A local convergence analysis for certain paradigm iterations is outlined as motivation for the use of these updates, and numerical experiments involving these iterations are discussed. Key words, least-change secant updates, quasi-Newton updates, parameter-dependent systems AMS(MOS) subject classification. 65H10 1. Introduction. Quasi-Newton methods are very widely used iterative methods for solving systems of nonlinear algebraic equations. The basic form of a quasi-Newton method for solving F(x) 0, F R n-. Rn, is (1.1) Xk+: =xk-B:F(xk), in which Bk ,. F(xk) E Rnn, the Jacobian (matrix) of F at xk. For practical success, it is usually essential to augment this basic form with procedures for modifying the step-B:F(xk) to ensure progress from bad starting points, but we need not consider such procedures here. For a general reference on all aspects of quasi-Newton methods, see Dennis and Schnabel [11]. The most effective quasi-Newton methods are those in which each successive Bk+: is determined as a least-change secant update of its predecessor Bk. As the name suggests, Bk+l is determined as a least-change secant update of Bk by making the least possible change in Bk (as measured by a suitable matrix norm) which incorporates current secant information (usually expressed in terms of successive x-and F-values) and other available information about the structure of F. There are also notable updates which, strictly speaking, are least-change inverse secant updates obtained in an analogous way by making the least possible change in B-:. When speaking generically of least-change secant updates, we intend to include these. In [10], Dennis and Schnabel precisely formalize the notion of a least-change secant update and show how the most widely used updates can be derived as least-change secant updates. In [12], Dennis and Walker show that least-change secant update methods, i.e., quasi-Newton methods which use least-change secant updates, have desirable convergence properties in general.

Using Taylor's series we propose a modified secant relation to get a more accurate approximation of the second curvature of the objective function. Then, based on this modified secant relation we present a new BFGS method for solving unconstrained optimization problems. The proposed method make use of both gradient and function values while the usual secant relation uses only gradient values. Under appropriate conditions, we show that the proposed method is globally convergent without needing convexity assumption on the objective function. Comparative results show computational efficiency of the proposed method in the sense of the Dolan-Moré performance profiles.

Using Taylor's series we propose a modified secant relation to get a more accurate approximation of the second curvature of the objective function. Then, based on this modified secant relation we present a new BFGS method for solving unconstrained optimization problems. The proposed method make use of both gradient and function values while the usual secant relation uses only gradient values. Under appropriate conditions, we show that the proposed method is globally convergent without needing convexity assumption on the objective function. Comparative results show computational efficiency of the proposed method in the sense of the Dolan-More performance profiles.

The Han-Powell method has proved to be extremely fast and robust for small optimum power flow problems (of the order of 100 buses) However, it balks at full size problems (of the order of 1000 buses) This paperdevelops a class of decompositions to break large problems down to sizes the Han-Powell algorithm can comfortably tackle From this class we select one member.called the super

The Han-Powell method has proved to be extremely fast and robust for small optimum power flow problems (of the order of 100 buses) However, it balks at full size problems (of the order of 1000 buses) This paperdevelops a class of decompositions to break large problems down to sizes the Han-Powell algorithm can comfortably tackle From this class we select one member.called the super

This paper presents a modified quasi-Newton method for structured unconstrained optimization. The usual SQN equation employs only the gradients, but ignores the available function value information. Several researchers paid attention to other secant conditions to get a better approximation of the Hessian matrix of the objective function. Recently Yabe et al. (2007) [6] proposed the modified secant condition which uses both gradient and function value information in order to get a higher-order accuracy in approximating the second curvature of the objective function. In this paper, we derive a new progressive modified SQN equation, with a vector parameter which use both available gradient and function value information, that maintains most properties of the usual and modified structured quasi-Newton methods. Furthermore, local and superlinear convergence of the algorithm is obtained under some reasonable conditions.

We apply and compare various artificial neural network (ANN) and other algorithms for the automated morphological classification of galaxies. The ANNs are presented here mathematically, as non-linear extensions of conventional statistical methods in astronomy. The methods are illustrated using a selection of subsets from the ESO-LV catalogue, for which both machine parameters and human classifications are available. The main methods we explore are: (i) principal component analysis (PCA), which provides information on how independent and informative the input parameters are; (ii) encoder neural networks, which allow us to find both linear (PCA-like) and non-linear combinations of the input, illustrating an example of an unsupervised ANN; and (iii) supervised ANNs (using the backpropagation or quasi-Newton algorithm) based on a training set for which the human classification is known. Here the output for previously unclassified galaxies can be interpreted as either a continuous (analogue) output (for example T-type) or a Bayesian a posteriori probability for each class. Although the ESO-LV parameters are suboptimal, the success of the ANN in reproducing the human classification is 2 T-type units, similar to the degree of agreement between two human experts who classify the same galaxy images on plate material. We also examine the aspects of ANN configurations, reproducibility, scaling of input parameters and redshift information.

SQOPT is a software package for minimizing a convex quadratic function subject to both equality and inequality constraints. SQOPT may also be used for linear programming and for finding a feasible point for a set of linear equalities and inequalities. SQOPT uses a two-phase, active-set, reduce-Hessian method. It is most efficient on problems with relatively few degrees of freedom (for example, if only some of the variables appear in the quadratic term, or the number of active constraints and bounds is nearly as large as the number of variables). However, unlike previous versions of SQOPT, there is no limit on the number of degrees of freedom. SQOPT is primarily intended for large linear and quadratic problems with sparse constraint matrices. A quadratic term 1 2 x T Hx in the objective function is represented by a user subroutine that returns the product Hx for a given vector x. SQOPT uses stable numerical methods throughout and includes a reliable basis package (for maintaining sparse LU factors of the basis matrix), a practical antidegeneracy procedure, scaling, and elastic bounds on any number of constraints and variables. SQOPT is part of the SNOPT package for large-scale nonlinearly constrained optimization. The source code is re-entrant and suitable for any machine with a Fortran compiler. SQOPT may be called from a driver program in Fortran, Matlab, oř C/Č++. SQOPT can also be used as a stand-alone package, reading data in the MPS format used by commercial mathematical programming systems.

SNOPT is a general-purpose system for solving optimization problems involving many variables and constraints. It minimizes a linear or nonlinear function subject to bounds on the variables and sparse linear or nonlinear constraints. It is suitable for large-scale linear and quadratic programming and for linearly constrained optimization, as well as for general nonlinear programs. SNOPT finds solutions that are locally optimal, and ideally any nonlinear functions should be smooth and users should provide gradients. It is often more widely useful. For example, local optima are often global solutions, and discontinuities in the function gradients can often be tolerated if they are not too close to an optimum. Unknown gradients are estimated by finite differences. SNOPT uses a sequential quadratic programming (SQP) algorithm that obtains search directions from a sequence of quadratic programming subproblems. Each QP subproblem minimizes a quadratic model of a certain Lagrangian function subject to a linearization of the constraints. An augmented Lagrangian merit function is reduced along each search direction to ensure convergence from any starting point. SNOPT is most efficient if only some of the variables enter nonlinearly, or if the number of active constraints (including simple bounds) is nearly as large as the number of variables. SNOPT requires relatively few evaluations of the problem functions. Hence it is especially effective if the objective or constraint functions (and their gradients) are expensive to evaluate. The source code for SNOPT is suitable for any machine with a Fortran compiler. SNOPT may be called from a driver program (typically in Fortran, C or MATLAB). SNOPT can also be used as a stand-alone package, reading data in the MPS format used by commercial mathematical programming systems.

A unifying approach is presented for the nonlinear static analysis of cable structures and for the form-finding of tensegrity structures. The novelty lies in the possibility of static analyses of structures where the stiffness matrix is singular throughout the path to equilibrium. The unification of static analyzes and form-finding procedures allows the understanding and treatment of tensegrity and cable structures as a single type of structure. A total potential energy function is derived in terms of nodal displacements which are the unknowns of a nonlinear programming problem. The proposed approach uses a Quasi-Newton method overcoming a limitation of the Newton Raphson Method employed by widel-used commercial Finite Element software packages. Example analyses are presented and compared with experimental results reported in the literature to demonstrate the feasibility of the proposed approach which is particularly useful for under-constrained structures that contain pre-tensioned elements.

We proposed a matrix-free direction with an inexact line search technique to solve system of nonlinear equations by using double direction approach. In this article, we approximated the Jacobian matrix by appropriately constructed matrix-free method via acceleration parameter. The global convergence of our method is established under mild conditions. Numerical comparisons reported in this paper are based on a set of large-scale test problems and show that the proposed method is efficient for large-scale problems.

A fast inversion technique for the interpretation of data from resistivity tomography surveys has been developed for operation on a microcomputer. This technique is based on the smoothness‐constrained least‐squares method and it produces a two‐dimensional subsurface model from the apparent resistivity pseudosection. In the first iteration, a homogeneous earth model is used as the starting model for which the apparent resistivity partial derivative values can be calculated analytically. For subsequent iterations, a quasi‐Newton method is used to estimate the partial derivatives which reduces the computer time and memory space required by about eight and twelve times, respectively, compared to the conventional least‐squares method. Tests with a variety of computer models and data from field surveys show that this technique is insensitive to random noise and converges rapidly. This technique takes about one minute to invert a single data set on an 80486DX microcomputer.

This research proposes a novel approach to bridging the gap between theoretical concepts and practical applications of inventory management functions within intelligent order management systems. The study introduces a robust framework specifically focused on optimizing inventory controls and provides a holistic solution for intelligent order management applications by harnessing fuzzy logic and rule-based algorithms. The proposed model streamlines the identification of demand and lead time patterns, thereby enhancing the intelligence and efficiency of existing order management systems. To validate the efficacy of this framework, it has been extensively tested within the operational context of prominent multinational B2C retailers. Results demonstrated the system's effectiveness in inventory management practices, particularly benefiting small to medium-sized enterprises. Furthermore, the scalable architecture of the system ensures further adaptability to diverse industries, including B2B distribution and manufacturing sectors, thus broadening its potential applications. This study significantly contributes to the advancements of intelligent order management offerings by optimizing inventory controls for businesses as they navigate through the complexities of modern supply chains.

This paper presents the application of Newton-based methods in the time-domain for the computation of the periodic steady state solutions of microgrids with multiple distributed generation units, harmonic stability and power quality analysis. Explicit representation of the commutation process of the power electronic converters and closed-loop power management strategies are fully considered. Case studies under different operating scenarios are presented: grid-connected mode, islanded mode, variations in the Thevenin equivalent of the grid and the loads. Besides, the close relation between the harmonic distortion, steady state performance of the control systems, asymptotic stability and power quality is analyzed in order to evaluate the importance and necessity of using full models in stressed and harmonic distorted scenarios.

This paper presents an introduction to the use of neural network computational algorithms for the identification of dynamic systems. Simulated linear and non-linear systems and real plant data are used to demonstrate the effectiveness of neural networks for the identification of neural transfer functions and to investigate the detection of the model structure, the influence of the size of the learning data set, the multiplicity of neural models and the effect of measurement noise. Backpropagation and quasi-Newton learning algorithms are discussed and compared. Neural networks offer the distinctive ability to learn very naturally complex relationships without requiring the explicit derivation of the model of the system.

KEY WORDS: topology optimization; dual methods; convex approximations; CONLIN; generalised MMA; quasi-Newton method; 1 INTRODUCTION As a result of several researches (e.g.[6, 9, 8, 12]), structural optimization problems with sizing or shape design variables can be solved efficiently by the mathematical programming approach and real-life applications can be handled. With the homogenization method proposed by Bendsøe and Kikuchi [2] and then further developed in several other works (see Bendsøe [1] for a review), structural optimization is now able to attack the topology design problem. Despite the fact that the material distribution problem looks like a sizing optimization, topology problems have its own characteristics and difficulties. The discretization of the material density introduces generally between 1000 and 10000 design variables, number that was never reached before. Furthermore, solving topology problems requires often a very high number of iterations to get a stationary distribution. It is usual to spend more than 100 iterations to solve the topology problem. Up to now, few solvers for huge optimization problems, other than simple optimality criteria, were available in topology. Thus, topology design was often restricted to formulations involving few design constraints. Generally, the topology design was formulated as a minimum problem of a stiffness criterion, like the compliance, with a bound over the volume. This communication wants to report how we improve the solution procedure of optimization of topology huge size problem by extending the mathematical programming approach that was successful for other structural optimization problems. At first we use dual methods that are well adapted to solve the convex separable optimization problems even if the number of design variables is large. On another hand, we look at the problem of selecting the most appropriate approximations. The choice of an approximation is a compromise between

This paper presents an improved diagonal Secant-like method using two-step approach for solving large scale systems of nonlinear equations. In this scheme, instead of using direct updating matrix in every iteration to construct the interpolation curves, we chose to use an implicit updating approach to obtain an enhanced approximation of the Jacobian matrix which only requires a vector storage. The fact that the proposed method solves systems of nonlinear equations without the cost of storing the weighted matrix can be considered as a clear advantage of this method over some variants of Newton's methods.

A new iterative method based on the quasi-Newton approach for solving systems of nonlinear equations, especially large scale is proposed. We used the weighted combination of the Trapezoidal and Simpson quadrature rules. Our goal is to enhance the efficiency of the well known Broyden method by reducing the number of iterations it takes to reach a solution. Local convergence analysis and computational results are given.

We present a new matrix-free method for the computation of negative curvature directions based on the eigenstructure of minimal-memory BFGS matrices. We determine via simple formulas the eigenvalues of these matrices and we compute the desirable eigenvectors by explicit forms. Consequently, a negative curvature direction is computed in such a way that avoids the storage and the factorization of any matrix. We propose a modification of the L-BFGS method in which no information is kept from old iterations, so that memory requirements are minimal. The proposed algorithm incorporates a curvilinear path and a linesearch procedure, which combines two search directions; a memoryless quasi-Newton direction and a direction of negative curvature. Results of numerical experiments for large scale problems are also presented.

We present a new matrix-free method for the computation of the negative curvature direction in large scale unconstrained problems. We describe a curvilinear method which uses a combination of a quasi-Newton direction and a negative curvature direction. We propose an algorithm for the computation of the search directions which uses information of two specific L-BFGS matrices in such a way that avoids both the calculation and the storage of the approximate Hessian. Explicit forms for the eigenpair that corresponds to the most negative eigenvalue of the approximate Hessian are also presented. Numerical results show that the proposed approach is promising.

In this work, we introduce a family of Least Change Secant Update Methods for solving Nonlinear Complementarity Problems based on its reformulation as a nonsmooth system using the one-parametric class of nonlinear complementarity functions introduced by Kanzow and Kleinmichel. We prove local and superlinear convergence for the algorithms. Some numerical experiments show a good performance of this algorithm.

In this work new quasi-Newton methods for solving large-scale nonlinear systems of equations are presented. In these methods q (¿ 1) columns of the approximation of the inverse Jacobian matrix are updated in such a way that the q last secant equations are satisÿed (whenever possible) at every iteration. An optimal maximum value for q that makes the method competitive is strongly suggested. The best implementation from the point of view of linear algebra and numerical stability is proposed and a local convergence result for the case q = 2 is proved. Several numerical comparative tests with other quasi-Newton methods are carried out.

This paper introduces new contrast functions for blind separation of sources with different time-frequency signatures. Two contrast functions based on the Kullback-Leibler and Jensen-Rényi divergences in the time-frequency (T-F) plane are introduced. Two iterative algorithms are proposed for the proposed contrasts optimization and source separation. One algorithm consists of spatial whitening and gradient-Jacobi maximization, combining Givens rotations and stochastic gradient. The second algorithm uses a quasi-Newton technique.

Uncertainty quantification of production forecasts is crucially important for business planning of hydrocarbon-field developments. This is still a very challenging task, especially when subsurface uncertainties must be conditioned to production data. Many different approaches have been proposed, each with their strengths and weaknesses. In this work, we develop a robust uncertainty-quantification work flow by seamless integration of a distributed-Gauss-Newton (GN) (DGN) optimization method with a Gaussian mixture model (GMM) and parallelized sampling algorithms. Results are compared with those obtained from other approaches. Multiple local maximum-a-posteriori (MAP) estimates are determined with the local-search DGN optimization method. A GMM is constructed to approximate the posterior probability-density function (PDF) by reusing simulation results generated during the DGN minimization process. The traditional acceptance/rejection (AR) algorithm is parallelized and applied to improve the quality of GMM samples by rejecting unqualified samples. AR-GMM samples are independent, identically distributed samples that can be directly used for uncertainty quantification of model parameters and production forecasts. The proposed method is first validated with 1D nonlinear synthetic problems with multiple MAP points. The AR-GMM samples are better than the original GMM samples. The method is then tested with a synthetic history-matching problem using the SPE01 reservoir model (Odeh 1981; Islam and Sepehrnoori 2013) with eight uncertain parameters. The proposed method generates conditional samples that are better than or equivalent to those generated by other methods, such as Markov-chain Monte Carlo (MCMC) and global-search DGN combined with the randomized-maximum-likelihood (RML) approach, but have a much lower computational cost (by a factor of five to 100). Finally, it is applied to a real-field reservoir model with synthetic data, with 235 uncertain parameters. A GMM with 27 Gaussian components is constructed to approximate the actual posterior PDF. There are 105 AR-GMM samples accepted from the 1,000 original GMM samples, and they are used to quantify the uncertainty of production forecasts. The proposed method is further validated by the fact that production forecasts for all AR-GMM samples are quite consistent with the production data observed after the history-matching period. The newly proposed approach for history matching and uncertainty quantification is quite efficient and robust. The DGN optimization method can efficiently identify multiple local MAP points in parallel. The GMM yields proposal candidates with sufficiently high acceptance ratios for the AR algorithm. Parallelization makes the AR algorithm much more efficient, which further enhances the efficiency of the integrated work flow.

Iterative methods have gained a solid reputation for efficient image restoration, for both spatially invariant and spatially variant blurs. This paper shows how a "strap-on" quasi-Newton Broyden method can further accelerate the convergence of these iterative methods with little extra overhead.

Iterative methods have gained a solid reputation for efficient image restoration, for both spatially invariant and spatially variant blurs. This paper shows how a “strap-on” quasi-Newton Broyden method can further accelerate the convergence of these iterative methods with little extra overhead.

Zontul, Metin (Arel Author)It is getting more difficult to retrieve relevant information regarding the user input query due to the large amount of information in the web. Unlike the conventional information retrieval (IR) algorithms, this study presents a new algorithm – reduced row echelon form IR method (rrefIR) – with higher average similarity precision to get more relevant and noise-free documents. For dimension reduction in the proposed algorithm, singular value decomposition (SVD) is applied on the reduced row echelon form – obtained by utilizing Gauss-Jordan method – of the covariance of term-document matrix (TDM). The rrefIR algorithm outperforms the LSI and COV algorithms with respect to Jaro-Winkler, Overlap, Tanimoto and Jaccard similarity measures in the means of average similarity precision. The physical reason for the better IR performance is the linear independent basis vectors set obtained by Gauss-Jordan operation. This basis set can be considered as the generating ...

In this paper, we present a hybrid optimization approach to solving the economic dispatch (ED) problem. The objective is to minimize the total fuel cost and keep the power flows within the security limits. The idea consists in combining principles from real coded genetic algorithms (RCGAs) [1] and a parallel quasi-Newton method (QN), in order to find a better compromise trade-off and a better precision while requiring a reasonable computing time. The approach that is presented utilizes both local and global optimization algorithms to find good design points more efficiently than either could. A parallel quasi-Newton method is used to explore the space of research and exploit all local information to progress towards a better point. The proposed approach improves the equality of the solution and speed of convergence of the algorithms. The hybrid methods developed is compared with a RCGAs and a classical methods of QN. The proposed approach has been tested on the IEEE 57 bus system [2].