geometric semantic genetic programming

A c c e p t e d M a n u s c r i p t • The behavior of a real Gas Turbine is modeled using GP • Several state-of-the-art GP algorithms are compared • Results show that standard GP with local search outperforms recent variants *Highlights (for review)

Ephemeral random constants are commonly used for symbolic regression with genetic programming. However, due to their random nature, it is difficult for genetic programming to find proper constants. This often leads to large and complex non-interpretable solutions. Therefore, in this short paper, we analyze a method that replaces ephemeral random constants with constant tokens whose values are optimized during evolution with the Sequential Least Squares Programming method. The results achieved on the studied regression problem indicate that replacing ephemeral random constants by optimized constant tokens leads to solutions with a significantly lower error which are also smaller than the solutions generated by a standard GP approach using ephemeral random constants. CCS CONCEPTS • Software and its engineering → Genetic programming; • Computing methodologies → Supervised learning by regression.

Moraglio et al. have recently introduced new genetic operators for genetic programming, called geometric semantic operators. These operators induce a unimodal fitness landscape for all the problems consisting in matching input data with known target outputs (like regression and classification). This feature facilitates genetic programming evolvability, which makes these operators extremely promising. Nevertheless, Moraglio et al. leave open problems, the most important one being the fact that these operators, by construction, always produce offspring that are larger than their parents, causing an exponential growth in the size of the individuals, which actually renders them useless in practice. In this paper we overcome this limitation by presenting a new efficient implementation of the geometric semantic operators. This allows us, for the first time, to use them on complex real-life applications, like the two problems in pharmacokinetics that we address here. Our results confirm the excellent evolvability of geometric semantic operators, demonstrated by the good results obtained on training data. Furthermore, we have also achieved a surprisingly good generalization ability, a fact that can be explained considering some properties of geometric semantic operators, which makes them even more appealing than before.

The canonical approach towards fitness evaluation in Genetic Programming (GP) is to use a static training set to determine fitness, based on a cost function averaged over all fitness-cases. However, motivated by different goals, researchers have recently proposed several techniques that focus selective pressure on a subset of fitness-cases at each generation. These approaches can be described as fitness-case sampling techniques, where the training set is sampled, in some way, to determine fitness. This paper shows a comprehensive evaluation of some of the most recent sampling methods, using benchmark and real-world problems for symbolic regression. The algorithms considered here are Interleaved Sampling, Random Interleaved Sampling, Lexicase Selection and a new sampling technique is proposed called Keep-Worst Interleaved Sampling (KW-IS). The algorithms are extensively evaluated based on test performance, overfitting and bloat. Results suggest that sampling techniques can improve performance compared with standard GP. While on synthetic benchmarks the difference is slight or none at all, on real-world problems the differences are substantial. Some of the best results were achieved by Lexicase Selection and Keep Worse-Interleaved Sampling. Results also show that on real-world problems overfitting correlates strongly with bloating. Furthermore, the sampling techniques provide efficiency, since they reduce the number of fitness-case evaluations required over an entire run.

Genetic programming (GP) is the subset of evolutionary computation in which the aim is to create executable programs. It is an exciting field with many applications, some immediate and practical, others long-term and visionary. In this chapter we provide a brief history of the ideas of genetic programming. We give a taxonomy of approaches and place genetic programming in a broader taxonomy of artificial intelligence. We outline some current research topics and point to successful use cases. We conclude with some practical GP-related resources including software packages and venues for GP publications.

Moraglio et al. have recently introduced new genetic operators for genetic programming, called geometric semantic operators. These operators induce a unimodal fitness landscape for all the problems consisting in matching input data with known target outputs (like regression and classification). This feature facilitates genetic programming evolvability, which makes these operators extremely promising. Nevertheless, Moraglio et al. leave open problems, the most important one being the fact that these operators, by construction, always produce offspring that are larger than their parents, causing an exponential growth in the size of the individuals, which actually renders them useless in practice. In this paper we overcome this limitation by presenting a new efficient implementation of the geometric semantic operators. This allows us, for the first time, to use them on complex real-life applications, like the two problems in pharmacokinetics that we address here. Our results confirm the excellent evolvability of geometric semantic operators, demonstrated by the good results obtained on training data. Furthermore, we have also achieved a surprisingly good generalization ability, a fact that can be explained considering some properties of geometric semantic operators, which makes them even more appealing than before.

Redistricting consists in dividing a geographic space or region of spatial units into smaller subregions or districts. In this paper, a Genetic Programming framework that addresses the electoral redistricting problem is proposed. The method uses new genetic operators, called geometric semantic genetic operators, that employ semantic information directly in the evolutionary search process with the objective of improving its optimization ability. The system is compared to several different redistricting techniques, including evolutionary and non-evolutionary methods. The simulations were made on ten real data-sets and, even though the studied problem does not belong to the classes of problems for which geometric semantic operators induce a unimodal fitness landscape, the results we present demonstrate the effectiveness of the proposed technique.

We present an approach for regression problems that employs analytic continued fractions as a novel representation. Comparative computational results using a memetic algorithm are reported in this work. Our experiments included fifteen other different machine learning approaches including five genetic programming methods for symbolic regression and ten machine learning methods. The comparison on training and test generalization was performed using 94 datasets of the Penn State Machine Learning Benchmark. The statistical tests showed that the generalization results using analytic continued fractions provides a powerful and interesting new alternative in the quest for compact and interpretable mathematical models for artificial intelligence.

The canonical approach towards fitness evaluation in Genetic Programming (GP) is to use a static training set to determine fitness, based on a cost function averaged over all fitness-cases. However, motivated by different goals, researchers have recently proposed several techniques that focus selective pressure on a subset of fitness-cases at each generation. These approaches can be described as fitness-case sampling techniques, where the training set is sampled, in some way, to determine fitness. This paper shows a comprehensive evaluation of some of the most recent sampling methods, using benchmark and real-world problems for symbolic regression. The algorithms considered here are Interleaved Sampling, Random Interleaved Sampling, Lexicase Selection and a new sampling technique is proposed called Keep-Worst Interleaved Sampling (KW-IS). The algorithms are extensively evaluated based on test performance, overfitting and bloat. Results suggest that sampling techniques can improve performance compared with standard GP. While on synthetic benchmarks the difference is slight or none at all, on real-world problems the differences are substantial. Some of the best results were achieved by Lexicase Selection and Keep Worse-Interleaved Sampling. Results also show that on real-world problems overfitting correlates strongly with bloating. Furthermore, the sampling techniques provide efficiency, since they reduce the number of fitness-case evaluations required over an entire run.

The canonical approach towards fitness evaluation in Genetic Programming (GP) is to use a static training set to determine fitness, based on a cost function averaged over all fitness-cases. However, motivated by different goals, researchers have recently proposed several techniques that focus selective pressure on a subset of fitness-cases at each generation. These approaches can be described as fitness-case sampling techniques, where the training set is sampled, in some way, to determine fitness. This paper shows a comprehensive evaluation of some of the most recent sampling methods, using benchmark and real-world problems for symbolic regression. The algorithms considered here are Interleaved Sampling, Random Interleaved Sampling, Lexicase Selection and a new sampling technique is proposed called Keep-Worst Interleaved Sampling (KW-IS). The algorithms are extensively evaluated based on test performance, overfitting and bloat. Results suggest that sampling techniques can improve performance compared with standard GP. While on synthetic benchmarks the difference is slight or none at all, on real-world problems the differences are substantial. Some of the best results were achieved by Lexicase Selection and Keep Worse-Interleaved Sampling. Results also show that on real-world problems overfitting correlates strongly with bloating. Furthermore, the sampling techniques provide efficiency, since they reduce the number of fitness-case evaluations required over an entire run.

In the Genetic Programming (GP) community there has been a great interest in developing semantic genetic operators. These type of operators use information of the phenotype to create offspring. The most recent approaches of semantic GP include the GP framework based on the alignment of error space, the geometric semantic genetic operators, and backpropagation genetic operators. Our contribution proposes two semantic operators based on projections in the phenotype space. The proposed operators have the characteristic, by construction, that the offspring's fitness is as at least as good as the fitness of the best parent; using as fitness the euclidean distance. The semantic operators proposed increment the learning capabilities of GP. These operators are compared against a traditional GP and Geometric Semantic GP in the Human oral bioavailability regression problem and 13 classification problems. The results show that a GP system with our novel semantic operators has the best performance in the training phase in all the problems tested.

The canonical approach towards fitness evaluation in Genetic Programming (GP) is to use a static training set to determine fitness, based on a cost function averaged over all fitness-cases. However, motivated by different goals, researchers have recently proposed several techniques that focus selective pressure on a subset of fitness-cases at each generation. These approaches can be described as fitness-case sampling techniques, where the training set is sampled, in some way, to determine fitness. This paper shows a comprehensive evaluation of some of the most recent sampling methods, using benchmark and real-world problems for symbolic regression. The algorithms considered here are Interleaved Sampling, Random Interleaved Sampling, Lexicase Selection and a new sampling technique is proposed called Keep-Worst Interleaved Sampling (KW-IS). The algorithms are extensively evaluated based on test performance, overfitting and bloat. Results suggest that sampling techniques can improve performance compared with standard GP. While on synthetic benchmarks the difference is slight or none at all, on real-world problems the differences are substantial. Some of the best results were achieved by Lexicase Selection and Keep Worse-Interleaved Sampling. Results also show that on real-world problems overfitting correlates strongly with bloating. Furthermore, the sampling techniques provide efficiency, since they reduce the number of fitness-case evaluations required over an entire run.

This work deals with the analysis and prediction of the behavior of a gas turbine (GT), the Mitsubishi single shaft Turbo-Generator Model MS6001, which has a 30 MW generation capacity. GTs such as this are of great importance in industry, as drivers of gas compressors for power generation. Because of their complexity and their execution environment, the failure rate of GTs can be high with severe consequences. These units are subjected to transient operations due to starts, load changes and sudden stops that degrade the system over time. To better understand the dynamic behavior of the turbine and to mitigate the aforementioned problems, these transient conditions need to be analyzed and predicted. In the absence of a thermodynamic mathematical model, other approaches should be considered to construct representative models that can be used for condition monitoring of the GT, to predict its behavior and detect possible system malfunctions. One way to derive such models is to use data-driven approaches based on machine learning and artificial intelligence. This work studies the use of state-of-the-art genetic programming (GP) methods to model the Mitsubishi single shaft Turbo-Generator. In particular, we evaluate and compare variants of GP and geometric semantic GP (GSGP) to build models that predict the fuel flow of the unit and the exhaust gas temperature. Results show that an algorithm, proposed by the authors, that integrates a local search mechanism with GP (GP-LS) outperforms all other state-of-the-art variants studied here on both problems, using real-world and representative data recorded during normal system operation. Moreover, results show that GP-LS outperforms seven other modeling techniques, including neural networks and isotonic regression, confirming the importance of GP-based algorithms in this domain.

Since its introduction, Geometric Semantic Genetic Programming (GSGP) has aroused the interest of numerous researchers and several studies have demonstrated that GSGP is able to effectively optimize training data by means of small variation steps, that also have the effect of limiting overfitting. In order to speed up the search process, in this paper we propose a system that integrates a local search strategy into GSGP (called GSGP-LS). Furthermore, we present a hybrid approach, that combines GSGP and GSGP-LS, aimed at exploiting both the optimization speed of GSGP-LS and the ability to limit over-fitting of GSGP. The experimental results we present, performed on a set of complex real-life applications, show that GSGP-LS achieves the best training fitness while converging very quickly, but severely overfits. On the other hand, GSGP converges slowly relative to the other methods, but is basically not affected by overfitting. The best overall results were achieved with the hybrid approach, allowing the search to converge quickly, while also exhibiting a noteworthy ability to limit overfitting. These results are encouraging, and suggest that future GSGP algorithms should focus on finding the correct balance between the greedy optimization of a local search strategy and the more robust geometric semantic operators.

The purpose of this study is to develop an innovative system for Coumarin-derived drug dosing, suitable for elderly patients. Recent research highlights that the pharmacological response of the patient is often affected by many exogenous factors other than the dosage prescribed and these factors could form a very complex relationship with the drug dosage. For this reason, new powerful computational tools are needed for approaching this problem. The system we propose is called Geometric Semantic Genetic Programming, and it is based on the use of recently defined geometric semantic genetic operators. In this paper, we present a new implementation of this Genetic Programming system, that allow us to use it for real-life applications in an efficient way, something that was impossible using the original definition. Experimental results show the suitability of the proposed system for managing anticoagulation therapy. In particular, results obtained with Geometric Semantic Genetic Programming are significantly better than the ones produced by standard Genetic Programming both on training and on out-of-sample test data.

Genetic programming (GP) is not a field noted for the rigor of its benchmarking. Some of its benchmark problems are popular purely through historical contingency, and they can be criticized as too easy or as providing misleading information concerning real-world performance, but they persist largely because of inertia and the lack of good alternatives. Even where the problems themselves are impeccable, comparisons between studies are made more difficult by the lack of standardization. We argue that the definition of standard benchmarks is an essential step in the maturation of the field. We make several contributions towards this goal. We motivate the development of a benchmark suite and define its goals; we survey existing practice; we enumerate many candidate benchmarks; we report progress on reference implementations; and we set out a concrete plan for gathering feedback from the GP community that would, if adopted, lead to a standard set of benchmarks.

We present the results of a community survey regarding genetic programming (GP) benchmark practices. Analysis shows broad consensus that improvement is needed in problem selection and experimental rigor. While views expressed in the survey dissuade us from proposing a large-scale benchmark suite, we find community support for creating a “blacklist” of problems which are in common use but have important flaws, and whose use should therefore be discouraged. We propose a set of possible replacement problems.

Multiclass classification is a common requirement of many land cover/land use applications, one of the pillars of land science studies. Even though genetic programming has been applied with success to a large number of applications, it is not particularly suited for multiclass classification, thus limiting its use on such studies. In this paper we take a step forward towards filling this gap, investigating the performance of recently defined geometric semantic operators on two land cover/land use multiclass classification problems and also on a benchmark problem. Our results clearly indicate that genetic programming using the new geometric semantic operators outperforms standard genetic programming for all the studied problems, both on training and test data.