Quasi-Newton methods for multiobjective optimization problems

Many difficulties such as multi-modality, dimensionality and differentiability are associated with the optimization of large-scale problems. Traditional techniques such as steepest decent, linear programing and dynamic programing generally fail to solve such large-scale problems especially with nonlinear objective functions. Most of the traditional techniques require gradient information and hence it is not possible to solve non-differentiable functions with the help of such traditional techniques. Moreover, such techniques often fail to solve optimization problems that have many local optima. To overcome these problems, there is a need to develop more powerful optimization techniques and research is going on to find effective optimization techniques since last three decades.

International Journal of Engineering Research and, 2015

Many difficulties are associated with the optimization of large-scale problems. The major difficulties are multi-modality, dimensionality and differentiability. Traditional techniques generally fail to solve such large-scale problems especially with nonlinear objective functions. The main problem is to solve non-differentiable functions with the help of traditional techniques because most of the traditional techniques require gradient information and hence it is not possible. Moreover, such techniques often fail to solve optimization problems that have many local optima. To overcome these problems, there is a need to develop more powerful optimization techniques. These techniques are known as modern optimization technique. In this Paper, the theory needed to understand the modern optimization techniques are explained. These modern techniques are used to solve linear, nonlinear, differential and non-differential optimization problems. Although various optimization methods have been proposed in recent years, but some more popular optimization techniques such as Genetic Algorithm, Simulated Annealing Ant colony method, Honey Bee Algorithm are presented here. The methods were broadly reviewed.

RUN beyond the metaphor: An efficient optimization algorithm based on Runge Kutta method, 2021

The optimization field suffers from the metaphor-based "pseudo-novel" or "fancy" optimizers. Most of these cliché methods mimic animals' searching trends and possess a small contribution to the optimization process itself. Most of these cliché methods suffer from the locally efficient performance, biased verification methods on easy problems, and high similarity between their components' interactions. This study attempts to go beyond the traps of metaphors and introduce a novel metaphor-free population-based optimization method based on the mathematical foundations and ideas of the Runge Kutta (RK) method widely well-known in mathematics. The proposed RUNge Kutta optimizer (RUN) was developed to deal with various types of optimization problems in the future. The RUN utilizes the logic of slope variations computed by the RK method as a promising and logical searching mechanism for global optimization. This search mechanism benefits from two active exploration and exploitation phases for exploring the promising regions in the feature space and constructive movement toward the global best solution. Furthermore, an enhanced solution quality (ESQ) mechanism is employed to avoid the local optimal solutions and increase convergence speed. The RUN algorithm's efficiency was evaluated by comparing with other metaheuristic algorithms in 50 mathematical test functions and four real-world engineering problems. The RUN provided very promising and competitive results, showing superior exploration and exploitation tendencies, fast convergence rate, and local optima avoidance. In optimizing the constrained engineering problems, the metaphor-free RUN demonstrated its suitable performance as well. The authors invite the community for extensive evaluations of this deep-rooted optimizer as a promising tool for real-world optimization. The source codes, supplementary materials, and guidance for the developed method will be publicly

Advances in Mathematics: Scientific Journal

We propose in this article a hybridization of the algorithm of the MOMA-Plus method and that of the Differential Evolution method. This hybridization consists of defining a simplex around an efficient solution generated by MOMA-plus and applying the Differential Evolution algorithm to find a better solution than that obtained by MOMA-plus. The results interpreted through a performance study of the solutions obtained on multiobjective optimization test problems show that this hybridization improves the convergence of the basic MOMA-plus algorithm. Moreover, a better complexity than that of basic MOMA-plus is obtained.

International Journal of Numerical Modelling: Electronic Networks, Devices and Fields, 2018

The usual quasi-Newton method at each iteration utilize the gradients and ignores the available function value information. In this paper, we employ Taylor formula and introduce a new quasi-Newton method, which uses both available gradient and function value information. This method approximates the Hessian matrix with excellent accuracy. The global convergence of these method associated to a general line search rule will be also shown. In addition, we will show that average performance of proposed algorithm is better than some proposed methods.

International Journal of Engineering Research and

One of the most basic concepts in humankind's life is the search for an optimal state. As long as the life continues, seeking for perfection in decision making of many areas is fundamental. The concept of optimization is considered as a main goal for a decision maker (DM) to achieve the best solution or most favorable set of solutions of one or more given criteria. In fact, most of real-world problems involve many correlated and often conflicted objectives that should be maximized or minimized to have the problem solved; such setting creates a harder situation for a DM to define all those contradicting objectives in terms of a one single objective following single-objective optimization (SOO) approach. To overcome this limitation, multi-objective optimization (MOO) becomes one of the recent optimization approaches to formulate decision making problems in a more realistic manner. As the ultimate goal of solving MOO problems is to find the optimal set of non-dominated solutions, which is called Paretooptimal set of Pareto-optimal solutions, using classical methods of exact, heuristics or metaheuristics methods become more complicated and cannot guarantee that those optimal solutions will be found. Therefore, many methods have been developed to facilitate the process of solving MOO problems with respect to the role of DM, due to his authority to give further preference information concerning the Pareto-optimal solutions. To this end, this study introduces a brief definition of MOO problem formulation, representation and solution; including analytical comparison of most common MOO problem solving methods in the literature. It's concluded that MOO methods tree could be classified into four main branches based on DM preference articulation: No preference, A-priori, Interactive, and A-posteriori preferences articulation of DM.

Industrial PID Controller Tuning, 2021

Application of the Multiobjective Approach 6.1 Comparison of the Methods to Obtain the Pareto Front 6.1.1 Performance Comparison of the Scalarization Methods In order to compare the efficiency of different linearization methods, different tests were performed on the normalized process given bŷ

Journal of Computational Chemistry, 1989

A new method for obtaining optimized parameters for semiempirical methods has been developed and applied to the modified neglect of diatomic overlap (MNDO) method. The method uses derivatives of calculated values for properties with respect to adjustable parameters to obtain the optimized values of parameters. The large increase in speed is a result of using a simple series expression for calculated values of properties rather than employing full semiempirical calculations. With this optimization procedure, the rate-determining step for parameterizing elements changes from the mechanics of parameterization to the assembling of experimental reference data.

Baghdad Science Journal, 2023

In this paper, two modifications for spectral quasi-Newton algorithm of type BFGS are imposed. In the first algorithm, named SQN EI , a certain spectral parameter is used in such a step for BFGS algorithm differs from other presented algorithms. The second algorithm, SQN Ev-Iv , has both new parameter position and value suggestion. In SQN EI and SQN Ev-Iv methods, the parameters are involved in a search direction after an approximated Hessian matrix is updated. It is provided that two methods are effective under some assumptions. Moreover, the sufficient descent property is proved as well as the global and superlinear convergence for SQN Ev-Iv and SQN EI. Both of them are superior the standard BFGS (QN BFGS) and previous spectral quasi-Newton (SQN LC). However, SQN Ev-Iv is outstanding SQN EI if it is convergent to the solution. This means that, two modified methods are in the race for the more efficiency method in terms less iteration numbers and consuming time in running CPU. Finally, numerical results are presented for the four algorithms by running list of test problems with inexact line search satisfying Armijo condition.

Optimization, 2012

This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.