Academia.eduAcademia.edu

Figure 12 – uploaded by Rafael Stubs Parpinelli

See full PDFdownloadDownload figure
Fig. 1. Family of estimating PSO methods Until now, many researchers have paid much attention to the issue, i.e. effectually obtaining the PSO with high search performance, and proposed a number of advanced algorithms to deal with it. These endeavors can be basically divided into two approaches: manual estimation and mechanical estimation shown in Figure 1.

Figure 1 Family of estimating PSO methods Until now, many researchers have paid much attention to the issue, i.e. effectually obtaining the PSO with high search performance, and proposed a number of advanced algorithms to deal with it. These endeavors can be basically divided into two approaches: manual estimation and mechanical estimation shown in Figure 1.

Related Figures (119)

Another important aspect is the necessity to set synchronization barriers. A barrier forces a thread to wait until all other threads of the same block reach the barrier. It helps to guarantee the correctness of the algorithm running on the GPU, but it can reduce the time performance. Furthermore, threads within a block can cooperate among themselves by sharing data through some shared memory and must synchronize their execution to coordinate the memory accesses (see Fig. 1). Although the GPUs are famous because of their
Another important aspect is the necessity to set synchronization barriers. A barrier forces a thread to wait until all other threads of the same block reach the barrier. It helps to guarantee the correctness of the algorithm running on the GPU, but it can reduce the time performance. Furthermore, threads within a block can cooperate among themselves by sharing data through some shared memory and must synchronize their execution to coordinate the memory accesses (see Fig. 1). Although the GPUs are famous because of their
The NVIDIA CUDA platform classify the NVIDIA GPUs using what they call Compute Capability as depicted in NVIDIA (2010b). The cards with double-precision floating-point numbers have Compute Capability 1.3 or 2.x. The cards with 2.x Capability can run up to 1,024 threads in a block and has 48 KB of shared memory space. The other ones only can execute 512 threads and have 16 KB of shared memory space.
The NVIDIA CUDA platform classify the NVIDIA GPUs using what they call Compute Capability as depicted in NVIDIA (2010b). The cards with double-precision floating-point numbers have Compute Capability 1.3 or 2.x. The cards with 2.x Capability can run up to 1,024 threads in a block and has 48 KB of shared memory space. The other ones only can execute 512 threads and have 16 KB of shared memory space.
this case, were have maintained the synchronization barriers only in the functions used to update the positions and evaluate the barycenter. The pseudocode of the Asynchronous FSS - Version A is shown in algorithm 2. In the second approach, called Asynchronous - Version B, all the synchronization barriers were removed from the code in order to have a full asynchronous version. The pseudocode of the Asynchronous FSS - Version B is shown in algorithm 3.
this case, were have maintained the synchronization barriers only in the functions used to update the positions and evaluate the barycenter. The pseudocode of the Asynchronous FSS - Version A is shown in algorithm 2. In the second approach, called Asynchronous - Version B, all the synchronization barriers were removed from the code in order to have a full asynchronous version. The pseudocode of the Asynchronous FSS - Version B is shown in algorithm 3.
5. Simulation setup and results The FSS versions detailed in the previous section were implemented on the CUDA Platform. In this section we present the simulations executed in order to evaluate the fitness performance of these different approaches. We also focused on the analysis of the execution time. Theory and New Applications of Swarm Intelligence
5. Simulation setup and results The FSS versions detailed in the previous section were implemented on the CUDA Platform. In this section we present the simulations executed in order to evaluate the fitness performance of these different approaches. We also focused on the analysis of the execution time. Theory and New Applications of Swarm Intelligence
In all these experiments we have used a combination of individual and volitive steps at both initial and final limits with a percentage of the function search space Bastos-Filho et al. (2009). Table 2 presents the used parameters for the steps (individual and volitive). Three We used a 1296 MHz GeForce GTX 280 with 240 Processing Cores to run the GPU-based FSS algorithms. All simulations were performed using 30 fish and we run 50 trial to evaluate the average fitness. All schools were randomly initialized in an area far from the optimal solution in every dimension. This allows a fair convergence analysis between the algorithms. All the random numbers needed by the FSS algorithm running on GPU were generated by a normal distribution using the proposal depicted in Bastos-Filho et al. (2010).
In all these experiments we have used a combination of individual and volitive steps at both initial and final limits with a percentage of the function search space Bastos-Filho et al. (2009). Table 2 presents the used parameters for the steps (individual and volitive). Three We used a 1296 MHz GeForce GTX 280 with 240 Processing Cores to run the GPU-based FSS algorithms. All simulations were performed using 30 fish and we run 50 trial to evaluate the average fitness. All schools were randomly initialized in an area far from the optimal solution in every dimension. This allows a fair convergence analysis between the algorithms. All the random numbers needed by the FSS algorithm running on GPU were generated by a normal distribution using the proposal depicted in Bastos-Filho et al. (2010).
Analyzing the convergence of the fitness values, the results for the parallel FSS versions on the GPU demonstrate that there are no reduction on the quality performance over the original version running on the CPU. Furthermore, there is a slight improvement in the quality of the values found for the Rastrigin function (see Fig. 4), specially for the asynchronous FSS version B. It might occurs because the outdated data generated by the race condition can avoid premature convergence to local minima in multimodal problems. Fig. 3. Rosenbrock’s fitness convergence as a function of the number of iterations.
Analyzing the convergence of the fitness values, the results for the parallel FSS versions on the GPU demonstrate that there are no reduction on the quality performance over the original version running on the CPU. Furthermore, there is a slight improvement in the quality of the values found for the Rastrigin function (see Fig. 4), specially for the asynchronous FSS version B. It might occurs because the outdated data generated by the race condition can avoid premature convergence to local minima in multimodal problems. Fig. 3. Rosenbrock’s fitness convergence as a function of the number of iterations.
Theory and New Applications of Swarm Intelligence
Theory and New Applications of Swarm Intelligence
Table 4. The Average Value and Standard Deviation of the Fitness value at the 10,000 iteration for Rosenbrock function.
Table 4. The Average Value and Standard Deviation of the Fitness value at the 10,000 iteration for Rosenbrock function.
Fig. 5. Griewank’s fitness convergence as a function of the number of iterations. According to these results, all FSS implementations based on the GPU achieved a time performance around 6 times better than the CPU version.
Fig. 5. Griewank’s fitness convergence as a function of the number of iterations. According to these results, all FSS implementations based on the GPU achieved a time performance around 6 times better than the CPU version.
Fig. 4. Rastrigin’s fitness convergence as a function of the number of iterations. Analysis of the Performance of the Fish School Search Algorithm Running in Graphic Processing Units
Fig. 4. Rastrigin’s fitness convergence as a function of the number of iterations. Analysis of the Performance of the Fish School Search Algorithm Running in Graphic Processing Units
‘ig. 1. Illustration of Probability Density Function for Gauss Distribution
‘ig. 1. Illustration of Probability Density Function for Gauss Distribution
Social Emotional Optimization Algorithm with Random Emotional Selection Strategy
Social Emotional Optimization Algorithm with Random Emotional Selection Strategy
Theory and New Applications of Swarm Intelligence
Theory and New Applications of Swarm Intelligence
Social Emotional Optimization Algorithm with Random Emotional Selection Strategy
Social Emotional Optimization Algorithm with Random Emotional Selection Strategy
Theory and New Applications of Swarm Intelligence
Theory and New Applications of Swarm Intelligence
Social Emotional Optimization Algorithm with Random Emotional Selection Strategy
Social Emotional Optimization Algorithm with Random Emotional Selection Strategy
Theory and New Applications of Swarm Intelligence
Theory and New Applications of Swarm Intelligence
As genetic operations in the RGA/E, roulette wheel selection, BLX-« crossover, random mutation, non-redundant strategy, and elitism strategy are used for efficiently finding an optimal individual (i.e. an optimal PSO) from the population of parameter values of the PSO. On being detailed, further refer to (33).
As genetic operations in the RGA/E, roulette wheel selection, BLX-« crossover, random mutation, non-redundant strategy, and elitism strategy are used for efficiently finding an optimal individual (i.e. an optimal PSO) from the population of parameter values of the PSO. On being detailed, further refer to (33).
Fig. 3. Comparison of two pairs of the used fitness functions As an example, Figure 3 illustrates the relative evaluation between two pairs of the criteria, {(4,), 3+ and {F,,F,}, during the evolutionary computation. It is clearly observed that the properties of the instantaneous fitness functions, g(q,) and g,, are quite different. Namely, while the change of g(q,) is monotonous increment, the change of g, is non-monotonous increment with violent stochastic vibration. In contrast to this, the criteria, F, and F,, are all monotonous increment with a minute vibration. Because both F, and F, are the sum of instantaneous fitness functions, g(q,) and %,, over time-step, in theory, their variance is inversely proportional to the interval of summation. Thus, they could lead to vastly inhibit noise which comes from dynamic evaluation to the estimation. This property indicates that which of both F, and F, is well suitable for evaluating the search performance of the PSO, regardless of the difference in objects of evaluation themselves.
Fig. 3. Comparison of two pairs of the used fitness functions As an example, Figure 3 illustrates the relative evaluation between two pairs of the criteria, {(4,), 3+ and {F,,F,}, during the evolutionary computation. It is clearly observed that the properties of the instantaneous fitness functions, g(q,) and g,, are quite different. Namely, while the change of g(q,) is monotonous increment, the change of g, is non-monotonous increment with violent stochastic vibration. In contrast to this, the criteria, F, and F,, are all monotonous increment with a minute vibration. Because both F, and F, are the sum of instantaneous fitness functions, g(q,) and %,, over time-step, in theory, their variance is inversely proportional to the interval of summation. Thus, they could lead to vastly inhibit noise which comes from dynamic evaluation to the estimation. This property indicates that which of both F, and F, is well suitable for evaluating the search performance of the PSO, regardless of the difference in objects of evaluation themselves.
Fig. 4. Fitness functions corresponding to the given benchmark problems in two-dimensional space. (a) The Sphere problem, (b) The Griewank problem, (c) The Rastrigin problem, (d) The Rosenbrock problem. Figure 4 illustrates the distribution of each fitness function in two-dimensional space. It is clearly shown that the properties of each problem, i.e. the Sphere problem is an unimodal with axes-symmetry, the Rosenbrock problem is an unimodal with axes-asymmetry, and the Griewank and Rastrigin problems are multimodal with different distribution density and axes-symmetry.
Fig. 4. Fitness functions corresponding to the given benchmark problems in two-dimensional space. (a) The Sphere problem, (b) The Griewank problem, (c) The Rastrigin problem, (d) The Rosenbrock problem. Figure 4 illustrates the distribution of each fitness function in two-dimensional space. It is clearly shown that the properties of each problem, i.e. the Sphere problem is an unimodal with axes-symmetry, the Rosenbrock problem is an unimodal with axes-asymmetry, and the Griewank and Rastrigin problems are multimodal with different distribution density and axes-symmetry.
Table 2. Estimated appropriate values of parameters in the PSO, cumulative fitness values, and frequencies in the top-twenty optimizers. The PSO in a-type: Cy) = 0, ¢; = 0, €y > 0; The PSO in b-type: €, = 0, ¢; > 0, 6 > 0; The PSO in c-type: ¢y > 0, ¢, = 0, €, > 0; in d-type: €) > 0,¢, > 0,é, > 0. The symbol ”-” signifies no result corresponding to contain type of the PSO. Theory and New Applications of Swarm Intelligence
Table 2. Estimated appropriate values of parameters in the PSO, cumulative fitness values, and frequencies in the top-twenty optimizers. The PSO in a-type: Cy) = 0, ¢; = 0, €y > 0; The PSO in b-type: €, = 0, ¢; > 0, 6 > 0; The PSO in c-type: ¢y > 0, ¢, = 0, €, > 0; in d-type: €) > 0,¢, > 0,é, > 0. The symbol ”-” signifies no result corresponding to contain type of the PSO. Theory and New Applications of Swarm Intelligence
Theory and New Applications of Swarm Intelligence
Theory and New Applications of Swarm Intelligence
Table 5. Estimated appropriate values of parameters in the PSO, criterion values and frequencies in the top-twentv optimizers. The PSO in d-tvne: 6. > 0.4, > 0.6, > 0. Theory and New Applications of Swarm Intelligence below.
Table 5. Estimated appropriate values of parameters in the PSO, criterion values and frequencies in the top-twentv optimizers. The PSO in d-tvne: 6. > 0.4, > 0.6, > 0. Theory and New Applications of Swarm Intelligence below.
Clans are groups of individuals united by a kinship and each clan has at least one guide. Inspired by these leadership characteristics, Carvalho and Bastos-Filho (Carvalho & Bastos-Filho, 2009a) proposed a topology called Clan PSO. Asa part of the topology, each clan is composed by particles which are connected to the other particles of the same clan in order to share information quickly, as shown in the example depicted in Figure 2. The algorithm has two phases executed in every iteration, the delegation of the leaders and the conference of the leaders. In previous works, Carvalho and Bastos-Filho (Carvalho & Bastos-Filho, 2009a) recommended to use just some few clans in order to avoid extra overhead within the iteration. More details about them are given in the following subsections. 3. Clan PSO
Clans are groups of individuals united by a kinship and each clan has at least one guide. Inspired by these leadership characteristics, Carvalho and Bastos-Filho (Carvalho & Bastos-Filho, 2009a) proposed a topology called Clan PSO. Asa part of the topology, each clan is composed by particles which are connected to the other particles of the same clan in order to share information quickly, as shown in the example depicted in Figure 2. The algorithm has two phases executed in every iteration, the delegation of the leaders and the conference of the leaders. In previous works, Carvalho and Bastos-Filho (Carvalho & Bastos-Filho, 2009a) recommended to use just some few clans in order to avoid extra overhead within the iteration. More details about them are given in the following subsections. 3. Clan PSO
After the Delegation, the leaders of each clan are selected and a new virtual swarm is composed by them. A PSO with the leaders can be ran using global (Figure 4(a)) or local (Figure 4(b)) topology. The former induces a faster convergence, while the latter allows more exploitation.
After the Delegation, the leaders of each clan are selected and a new virtual swarm is composed by them. A PSO with the leaders can be ran using global (Figure 4(a)) or local (Figure 4(b)) topology. The former induces a faster convergence, while the latter allows more exploitation.
At each iteration, each clan singly performs an independent PSO and the particle that obtainec the best position within the clan is delegated as a leader of the clan in the iteration. Figure ‘ illustrates an example with the leader delegated in each clan.
At each iteration, each clan singly performs an independent PSO and the particle that obtainec the best position within the clan is delegated as a leader of the clan in the iteration. Figure ‘ illustrates an example with the leader delegated in each clan.
Volitive PSO is a hybridization of the FSS and the PSO algorithms and it was proposed by (Cavalcanti-Junior et al., 2011). The algorithm uses two FSS operators in the Inertia PSO, the feeding and the collective-volitive movement. Each particle becomes a weighted particle, where the weights are used to define the collective-volitive movement, resulting in an expansion or contraction of the school. As a results, the Volitive PSO presents good features of the PSO and the FSS to tackle dynamic problems. These features are, respectively, fast convergence and the capacity to self-regulate the granularity of the search by using the volitive operator. Figure 5 illustrates the features aggregated in the Volitive PSO.
Volitive PSO is a hybridization of the FSS and the PSO algorithms and it was proposed by (Cavalcanti-Junior et al., 2011). The algorithm uses two FSS operators in the Inertia PSO, the feeding and the collective-volitive movement. Each particle becomes a weighted particle, where the weights are used to define the collective-volitive movement, resulting in an expansion or contraction of the school. As a results, the Volitive PSO presents good features of the PSO and the FSS to tackle dynamic problems. These features are, respectively, fast convergence and the capacity to self-regulate the granularity of the search by using the volitive operator. Figure 5 illustrates the features aggregated in the Volitive PSO.
Fig. 9. Box plot of Volitive Clan PSO 3x18 in the last iteration in high severity environment.
Fig. 9. Box plot of Volitive Clan PSO 3x18 in the last iteration in high severity environment.
Theory and New Applications of Swarm Intelligence
Theory and New Applications of Swarm Intelligence
(b) High severity with initialization of 50% o! particles.
(b) High severity with initialization of 50% o! particles.
(b) High severity with initialization of 50% o! particles.
(b) High severity with initialization of 50% o! particles.
Theory and New Applications of Swarm Intelligence
Theory and New Applications of Swarm Intelligence
Fig. 1. Iustration of neighbourhood topologies from [Mendes et al., 2004]: Fully connected (All), Ring, Four clusters, Pyramid and Square. where x,,(t) is the ith component of the position of the particle i and v,;(t) the ith component of its velocity; p,, is the iz, component of the best position ever visited by the ith particle; 9); is the im component of the best position ever visited by the neighbourhood of the particle; ais called inertia weight, it is used to control the impact of the previous history of velocity on the current one; 1 and r2 are uniformly distributed random numbers between 0 and 1; ci and cz are positive acceleration constants. The formula (4) is used for each dimension of the objective function, for each particle and synchronously at time step for all the particles of the swarm.
Fig. 1. Iustration of neighbourhood topologies from [Mendes et al., 2004]: Fully connected (All), Ring, Four clusters, Pyramid and Square. where x,,(t) is the ith component of the position of the particle i and v,;(t) the ith component of its velocity; p,, is the iz, component of the best position ever visited by the ith particle; 9); is the im component of the best position ever visited by the neighbourhood of the particle; ais called inertia weight, it is used to control the impact of the previous history of velocity on the current one; 1 and r2 are uniformly distributed random numbers between 0 and 1; ci and cz are positive acceleration constants. The formula (4) is used for each dimension of the objective function, for each particle and synchronously at time step for all the particles of the swarm.
Abscissa (m) of the used sensors function to the metrology set
Abscissa (m) of the used sensors function to the metrology set
Fig. 4. Half truss structure with its loads; Elpeam and Elcotumn are the unknown parameters.
Fig. 4. Half truss structure with its loads; Elpeam and Elcotumn are the unknown parameters.
Fig. 6. FE model of the physical model with the main beam bearing on SB and the wood plate. A 3D finite element model, 3DFEM, presents the global experience fig. 6. This 3DFEM helps to estimate the equivalent stiffnesses of the elastic support of the main beam considering the association of the stiffnesses of the SB and the plate as a Winkler spring of which stiffness by support is unknown. The fig. 7 gives the displacements measured during the experimental tests for each support set (illustrated by the points), whereas the displacements obtained with 3DFEM for each support set is illustrated by the continuous curve. The good correlation between measurements and simulations confirms the good quality of the 3DFEM model and justifies both a priory estimation of the equivalent stiffness by support and the imits of the space research using for the PSO.
Fig. 6. FE model of the physical model with the main beam bearing on SB and the wood plate. A 3D finite element model, 3DFEM, presents the global experience fig. 6. This 3DFEM helps to estimate the equivalent stiffnesses of the elastic support of the main beam considering the association of the stiffnesses of the SB and the plate as a Winkler spring of which stiffness by support is unknown. The fig. 7 gives the displacements measured during the experimental tests for each support set (illustrated by the points), whereas the displacements obtained with 3DFEM for each support set is illustrated by the continuous curve. The good correlation between measurements and simulations confirms the good quality of the 3DFEM model and justifies both a priory estimation of the equivalent stiffness by support and the imits of the space research using for the PSO.
Fig. 8. Localisation of the sensors used in both numerical and physical models.
Fig. 8. Localisation of the sensors used in both numerical and physical models.
Table 2. Abscissa of the sensors used during the real experience.
Table 2. Abscissa of the sensors used during the real experience.
Table 5. Identified equivalent stiffnesses using sensors from set B. Table 4. Identified equivalent stiffnesses using sensors from set A. 3.3 Sources of errors impacting the accuracy of the identification process: Synthetic experience Concerning synthetic experience, the following works relies on either numerical using the finite element (FE) software Castem©, or analytical model.
Table 5. Identified equivalent stiffnesses using sensors from set B. Table 4. Identified equivalent stiffnesses using sensors from set A. 3.3 Sources of errors impacting the accuracy of the identification process: Synthetic experience Concerning synthetic experience, the following works relies on either numerical using the finite element (FE) software Castem©, or analytical model.
stiffnesses were correctly identified but the identified parameters show slight variations for each of the 20 simulations. The results are presented in Table 6.
stiffnesses were correctly identified but the identified parameters show slight variations for each of the 20 simulations. The results are presented in Table 6.
The average value of each unknown parameter is very close to the reference but the standard deviation in not zero, which means that all solutions are not identical, even in this perfect case. Some scatter due to the meta-heuristic algorithm affects the identification process. However this scatter remains small. In real cases, it will be overshadowed by other error sources that will be studied now. Table 6. Statistical analysis of the identified parameters using exact field data, structure 1, metrology set CMi.
The average value of each unknown parameter is very close to the reference but the standard deviation in not zero, which means that all solutions are not identical, even in this perfect case. Some scatter due to the meta-heuristic algorithm affects the identification process. However this scatter remains small. In real cases, it will be overshadowed by other error sources that will be studied now. Table 6. Statistical analysis of the identified parameters using exact field data, structure 1, metrology set CMi.
Theory and New Applications of Swarm Intelligence
Theory and New Applications of Swarm Intelligence
Table 8. Statistical analysis of the identified parameters using noised field data, structure 3. he studied structure is here the structure 2, STR2, ie. the half truss structure presented at ection 3.1.1. Let us assume that one must identify the Young Modulus E, the inertia of the olumn I-olumn and the inertia of the beam, Ibeam. Four different levels of noise on field data € = 0% (perfect data), then ¢ = 1%, 3% and 5%) are considered and simulations are epeated 20 times. The results are presented in Figures 1land 12and Table 8.The first result s that one obtains a front of solutions, since it is not possible to uncouple the weight of E rom that of inertia: for the same product Elj, there exists an infinite number of acceptable airs {E, I; = (El))/E = k/E} satisfying the same criteria. In order to estimate the sensitivity of he identified parameters to the field data, the sensibility of the displacement to stiffnesses vas calculated. Elcotumn Or Elbeam are varied in the [-50%; +50%] range and the displacement s calculated on 3 points of the beam, and on three points of the column, cf. fig. 4. Only lisplacements perpendicular to the main axis of the element are calculated.
Table 8. Statistical analysis of the identified parameters using noised field data, structure 3. he studied structure is here the structure 2, STR2, ie. the half truss structure presented at ection 3.1.1. Let us assume that one must identify the Young Modulus E, the inertia of the olumn I-olumn and the inertia of the beam, Ibeam. Four different levels of noise on field data € = 0% (perfect data), then ¢ = 1%, 3% and 5%) are considered and simulations are epeated 20 times. The results are presented in Figures 1land 12and Table 8.The first result s that one obtains a front of solutions, since it is not possible to uncouple the weight of E rom that of inertia: for the same product Elj, there exists an infinite number of acceptable airs {E, I; = (El))/E = k/E} satisfying the same criteria. In order to estimate the sensitivity of he identified parameters to the field data, the sensibility of the displacement to stiffnesses vas calculated. Elcotumn Or Elbeam are varied in the [-50%; +50%] range and the displacement s calculated on 3 points of the beam, and on three points of the column, cf. fig. 4. Only lisplacements perpendicular to the main axis of the element are calculated.
Fig. 10. Illustration of the noise on field data and the dispersion of identified parameters k,, Structure 1, CM}.
Fig. 10. Illustration of the noise on field data and the dispersion of identified parameters k,, Structure 1, CM}.
The results confirm that only some field data are sensitive to the stiffness variations. The stiffness variation of the column inertia (reversely beam) has only a negligible influence on beam (reversely column) displacements. Thus, during the identification process, the magnitude of the errors on sensors localized on the beam will not impact the column because a lack of sensibility, and reversely. A more detailed analysis Table 9 shows that sensitivity of column displacement to column parameters is slightly larger than the same for the beam. The sensitivity has been calculated as the ratio between the variation of the displacements at the studied point with the variation of the stiffness. Those results show that the displacements of the column are more sensitive to a variation of the stiffness of the column that for the beam and explain why the scatter of the identified stiffnesses Elcolumnfig.12 is more important that the scatter of the identified stiffnesses Elbeam. Indeed,
The results confirm that only some field data are sensitive to the stiffness variations. The stiffness variation of the column inertia (reversely beam) has only a negligible influence on beam (reversely column) displacements. Thus, during the identification process, the magnitude of the errors on sensors localized on the beam will not impact the column because a lack of sensibility, and reversely. A more detailed analysis Table 9 shows that sensitivity of column displacement to column parameters is slightly larger than the same for the beam. The sensitivity has been calculated as the ratio between the variation of the displacements at the studied point with the variation of the stiffness. Those results show that the displacements of the column are more sensitive to a variation of the stiffness of the column that for the beam and explain why the scatter of the identified stiffnesses Elcolumnfig.12 is more important that the scatter of the identified stiffnesses Elbeam. Indeed,
Fig. 12. Inertia and Young Modulus identified, structure 2, noised with € = 3%. when field data on the column are noised, the inertia of the column to identified have to be badly identified from the reference value regards the magnitude of the noise. That result focuses how chosen of both the nature and the localisation of the field data regards the parameters to identified is important on the accuracy of the identification process.
Fig. 12. Inertia and Young Modulus identified, structure 2, noised with € = 3%. when field data on the column are noised, the inertia of the column to identified have to be badly identified from the reference value regards the magnitude of the noise. That result focuses how chosen of both the nature and the localisation of the field data regards the parameters to identified is important on the accuracy of the identification process.
We will illustrate the multi-objective particle swarm optimization for the design of a wood- plastic composite decking with three objectives [Michaud et al, 2009]. In this example, the optimization focuses on the creep, swelling, and exhaustion of abiotic resources functions. The design variables are mainly characteristics of raw materials such as timber particle sizes and chemical or thermal timber changes.
We will illustrate the multi-objective particle swarm optimization for the design of a wood- plastic composite decking with three objectives [Michaud et al, 2009]. In this example, the optimization focuses on the creep, swelling, and exhaustion of abiotic resources functions. The design variables are mainly characteristics of raw materials such as timber particle sizes and chemical or thermal timber changes.
Fig. 14. The influence graph of relationships between the decision variables and the objectives. with an intermediate value calculated at 0.69 (partially oriented) - see Michaud et al, op.cit., whereas the other variables used in the equations (12), (13) and (14) are continuous. In fact the def function (equation 12), in its developed formula has an algorithm form due to the conditions on the discrete kcran.
Fig. 14. The influence graph of relationships between the decision variables and the objectives. with an intermediate value calculated at 0.69 (partially oriented) - see Michaud et al, op.cit., whereas the other variables used in the equations (12), (13) and (14) are continuous. In fact the def function (equation 12), in its developed formula has an algorithm form due to the conditions on the discrete kcran.
Table 10. Variables X = {x1, x2, ..., X12} related to the composite formulation. see table 10 for the meaning of other variables. Water swelling function (SW)
Table 10. Variables X = {x1, x2, ..., X12} related to the composite formulation. see table 10 for the meaning of other variables. Water swelling function (SW)
Fig. 15. Solution space of the multi-objective (def, SW and efr) optimization problem determined from a MOPSO of 1000 generations of 30 particles. The equations (5) and (8) are used as position updating rule of respectively real and discrete variables. The equation (4) is used as velocity updating rule for all variables. During the optimization process, the real variables converge to their optima according to the objective functions, whereas each discrete variable randomly traverses its space of definition and consequently its best solution is identified. Due to the discrete variables, the solution space of the multi-objective optimization problem is discontinuous (fig. 15)
Fig. 15. Solution space of the multi-objective (def, SW and efr) optimization problem determined from a MOPSO of 1000 generations of 30 particles. The equations (5) and (8) are used as position updating rule of respectively real and discrete variables. The equation (4) is used as velocity updating rule for all variables. During the optimization process, the real variables converge to their optima according to the objective functions, whereas each discrete variable randomly traverses its space of definition and consequently its best solution is identified. Due to the discrete variables, the solution space of the multi-objective optimization problem is discontinuous (fig. 15)
Fig. 17. Stability of the Pareto front: a) constant number of particles, b) constant number of iterations [Ndiaye et al., 2009]. Theory and New Applications of Swarm Intelligence
Fig. 17. Stability of the Pareto front: a) constant number of particles, b) constant number of iterations [Ndiaye et al., 2009]. Theory and New Applications of Swarm Intelligence
‘he number of MOPSO solutions on composite formulations depends on the ratios between he components and their desired precision. The number of solutions grows in function with he precision of ratios between the components using a logarithm-like law. It starts at 1500 olutions for a precision of 2 (the lowest possible precision) to more than 5000 for a precision reater than 5 (fig.18). The matrix ratio in composite formulation generates a peak of olutions around 75% for any precision of ratio. This large number of solutions makes them lifficult to handle. One solution is to take into account, in the system process, the user of the ystem so he could fix the precision of ratios, and for each ratio, its desired range; the latter eing included in the domain of validity of the variable representative of the ratio. For xample if you want to formulate a wood-plastic composite with a matrix ratio lying etween 30% and 40% without biopolymer, it is sufficient to restrict the range of the variable epresentative of the matrix ratio (Am) between 0.3 and 0.4 and the one representative of the iopolymer ratio in matrix component (@pio) between 0.0 and 0.0. In this case the number of olutions in the Pareto front fall down to 20.
‘he number of MOPSO solutions on composite formulations depends on the ratios between he components and their desired precision. The number of solutions grows in function with he precision of ratios between the components using a logarithm-like law. It starts at 1500 olutions for a precision of 2 (the lowest possible precision) to more than 5000 for a precision reater than 5 (fig.18). The matrix ratio in composite formulation generates a peak of olutions around 75% for any precision of ratio. This large number of solutions makes them lifficult to handle. One solution is to take into account, in the system process, the user of the ystem so he could fix the precision of ratios, and for each ratio, its desired range; the latter eing included in the domain of validity of the variable representative of the ratio. For xample if you want to formulate a wood-plastic composite with a matrix ratio lying etween 30% and 40% without biopolymer, it is sufficient to restrict the range of the variable epresentative of the matrix ratio (Am) between 0.3 and 0.4 and the one representative of the iopolymer ratio in matrix component (@pio) between 0.0 and 0.0. In this case the number of olutions in the Pareto front fall down to 20.
PUCHY NWIELa-PICUloue AIQOTUTTT for Training the Radial Basis Function Network for Data Classification and Disease Diagnosis final outputs of network in the output layer. The input layer of this network has m units for m dimensional input vectors. The input units are fully connected to I hidden layer units, which are in turn fully connected to the J output layer units, where J is the number of output layer. Each neuron of the hidden layer has a parameter mean vector called center. Figure 1 shows the detailed structure of an RBF network. Each input data x with m dimensions, x= (X1,X9,...,X,,), are located in the input layer, which broadcast to hidden layer. The hidden layer has J neurons and each neuron compute the distance between the centers and the inputs. Each activation function of the neuron in hidden layer is chosen to be Gaussians and is characterized by their mean vectors c;and its spread parameter @; (i=1,2,...,]). That is, the activation function ¢(x) of the i hidden unit for an input vector x is given by:
PUCHY NWIELa-PICUloue AIQOTUTTT for Training the Radial Basis Function Network for Data Classification and Disease Diagnosis final outputs of network in the output layer. The input layer of this network has m units for m dimensional input vectors. The input units are fully connected to I hidden layer units, which are in turn fully connected to the J output layer units, where J is the number of output layer. Each neuron of the hidden layer has a parameter mean vector called center. Figure 1 shows the detailed structure of an RBF network. Each input data x with m dimensions, x= (X1,X9,...,X,,), are located in the input layer, which broadcast to hidden layer. The hidden layer has J neurons and each neuron compute the distance between the centers and the inputs. Each activation function of the neuron in hidden layer is chosen to be Gaussians and is characterized by their mean vectors c;and its spread parameter @; (i=1,2,...,]). That is, the activation function ¢(x) of the i hidden unit for an input vector x is given by:
where the weight matrix is represented as W and the ¢ matrix is the H matrix, respectively. The GD algorithm can be implemented to minimize the MSE term defined as the equation (3) based on the following equations. GD is the derivative based optimization algorithm (Karayiannis, 1999)that is used to search for the local minimum of a function. The algorithm takes steps proportional to negative of the gradient of function at the current situation with given the parameters a; and assumed all £; are equal to 0. In general, the output of a RBF network can be written in the following form.
where the weight matrix is represented as W and the ¢ matrix is the H matrix, respectively. The GD algorithm can be implemented to minimize the MSE term defined as the equation (3) based on the following equations. GD is the derivative based optimization algorithm (Karayiannis, 1999)that is used to search for the local minimum of a function. The algorithm takes steps proportional to negative of the gradient of function at the current situation with given the parameters a; and assumed all £; are equal to 0. In general, the output of a RBF network can be written in the following form.
Table 5. The used datasets in this study Table 4. The used parameters of Firefly algorithm
Table 5. The used datasets in this study Table 4. The used parameters of Firefly algorithm
Table 1. The used parameters of GA PUCHY WIReLa-TICUlouG AUIQOTUTTT for Training the Radial Basis Function Network for Data Classification and Disease Diagnosis
Table 1. The used parameters of GA PUCHY WIReLa-TICUlouG AUIQOTUTTT for Training the Radial Basis Function Network for Data Classification and Disease Diagnosis
Table 6. The average PCC and standard deviation results of the Iris dataset using different algorithm. One of the most important issues of designing the RBF network is the number of neurons in the hidden layer. Thus, we implement the RBF networks which have 1 neuron to 8 neurons for comparison, and each dataset is running 10 times based on 4-flod cross-validation. The average percentage and the corresponding standard derivation defined as the Eq. (25) of the designed RBF network by different algorithms are listed in Tables 6-10.
Table 6. The average PCC and standard deviation results of the Iris dataset using different algorithm. One of the most important issues of designing the RBF network is the number of neurons in the hidden layer. Thus, we implement the RBF networks which have 1 neuron to 8 neurons for comparison, and each dataset is running 10 times based on 4-flod cross-validation. The average percentage and the corresponding standard derivation defined as the Eq. (25) of the designed RBF network by different algorithms are listed in Tables 6-10.
Table 7. Statistical average PCC results of the Wine dataset using different algorithms
Table 7. Statistical average PCC results of the Wine dataset using different algorithms
Table 8. Statistical average PCC results of the Glass dataset using different algorithms. PICHy MeClta-MeurlsuC AIQOMUINT! for Training the Radial Basis Function Network for Data Classification and Disease Diagnosis
Table 8. Statistical average PCC results of the Glass dataset using different algorithms. PICHy MeClta-MeurlsuC AIQOMUINT! for Training the Radial Basis Function Network for Data Classification and Disease Diagnosis
Table 10. Statistical average results of the WDBC dataset using different algorithms.
Table 10. Statistical average results of the WDBC dataset using different algorithms.
Fig. 3. The mean square error versus complexity of the classification of the Iris dataset. Generally speaking, the complexity of trained RBF network with a large number of hidden nodes is larger but its corresponding mean square error is smaller. In experiments, The Figs. 3-7 recorded the mean square error and complexity of each trained RBF network based the Eq. (23) and (24). These figures clearly appear the phenomenon that the GD is the worst because of the largest mean square error with the same complexity among all algorithms.
Fig. 3. The mean square error versus complexity of the classification of the Iris dataset. Generally speaking, the complexity of trained RBF network with a large number of hidden nodes is larger but its corresponding mean square error is smaller. In experiments, The Figs. 3-7 recorded the mean square error and complexity of each trained RBF network based the Eq. (23) and (24). These figures clearly appear the phenomenon that the GD is the worst because of the largest mean square error with the same complexity among all algorithms.
Fig. 5. The mean square error versus complexity of the Glass classification.
Fig. 5. The mean square error versus complexity of the Glass classification.
Figure 3 shows the relationship between the complexity and mean square error in training the RBF networks of Iris dataset. These figures appear that the results of PSO, ABC and FF are superior to the GD. The Fig. 4 show the results of ABC and FF algorithm are superior to the results of GD and PSO in the training the RBF networks for Wine dataset. The Fig. 5 show the best is the result designed by PSO algorithm. The Fig. 7 demonstrates the best are the usages of the FF, however, the results of GD, PSO, ABC and FF do not clearly differentiate form the results of Fig. 6.
Figure 3 shows the relationship between the complexity and mean square error in training the RBF networks of Iris dataset. These figures appear that the results of PSO, ABC and FF are superior to the GD. The Fig. 4 show the results of ABC and FF algorithm are superior to the results of GD and PSO in the training the RBF networks for Wine dataset. The Fig. 5 show the best is the result designed by PSO algorithm. The Fig. 7 demonstrates the best are the usages of the FF, however, the results of GD, PSO, ABC and FF do not clearly differentiate form the results of Fig. 6.
Fig. 7. The mean square error v.s. complexity of the WDBC classification. Theory and New Applications of Swarm Intelligence
Fig. 7. The mean square error v.s. complexity of the WDBC classification. Theory and New Applications of Swarm Intelligence
Table 11. Area under curve (AUC) of ROC analysis of RBF network with eight hidden nodes. (The best results are highlighted in bold)
Table 11. Area under curve (AUC) of ROC analysis of RBF network with eight hidden nodes. (The best results are highlighted in bold)
In this paper we focused on the selection of small subsets of features from large-scale datasets. In this context, we mainly used two datasets generated using microarray data from breast cancer and leukemia that had been previously reported and studied in a similar fashion (Trevino and Falciani, 2006). For comparison with other PSO algorithms, we also used other three datasets in order to show overall effects and the generality of our adaptations (see Table 1).
In this paper we focused on the selection of small subsets of features from large-scale datasets. In this context, we mainly used two datasets generated using microarray data from breast cancer and leukemia that had been previously reported and studied in a similar fashion (Trevino and Falciani, 2006). For comparison with other PSO algorithms, we also used other three datasets in order to show overall effects and the generality of our adaptations (see Table 1).
Table 2. Comparison of PSO algorithms: the accuracy (Acc) and number of feature (size) of the best model found among 100 runs is shown in best-model columns. accuracy is the number of correctly predicted samples divided by the total. The mean and standard deviation (SD) of the accuracy and number of features were estimated from the resulted models (100 in most runs). Compared to uuPSO, all model accuracies and number of features were significant using a Wilcoxon rank sum test at p < 0.0005 except those marked with ¢ where p > 0.05. time was estimated for the 100 runs. A star (*) marks times estimated from partial results where at least 20 runs have finished. This estimated time should not be largely affected since run-time standard deviation is low. For example, for the ovarian dataset the standard deviation per run was 0.138 and 0.696 hours for sbPSO and Wang PSO respectively. The accuracy using all features is shown in column “All Acc” for comparison. Xu PSO was ran at f = 0.1, 0.3, 0.5, 0.7, and 0.9. The best result is shown. In four datasets f= 0.5 was the best in terms of accuracy and number of features except for the leukemia Golub dataset where f = 0.7 was better. Best results are shown in bold. value from an f-test. This forward-selection strategy is similar to that in PAM (Tibshirani et al., 2002) and the Prophet tool in GEPAS (Montaner et al., 2006).
Table 2. Comparison of PSO algorithms: the accuracy (Acc) and number of feature (size) of the best model found among 100 runs is shown in best-model columns. accuracy is the number of correctly predicted samples divided by the total. The mean and standard deviation (SD) of the accuracy and number of features were estimated from the resulted models (100 in most runs). Compared to uuPSO, all model accuracies and number of features were significant using a Wilcoxon rank sum test at p < 0.0005 except those marked with ¢ where p > 0.05. time was estimated for the 100 runs. A star (*) marks times estimated from partial results where at least 20 runs have finished. This estimated time should not be largely affected since run-time standard deviation is low. For example, for the ovarian dataset the standard deviation per run was 0.138 and 0.696 hours for sbPSO and Wang PSO respectively. The accuracy using all features is shown in column “All Acc” for comparison. Xu PSO was ran at f = 0.1, 0.3, 0.5, 0.7, and 0.9. The best result is shown. In four datasets f= 0.5 was the best in terms of accuracy and number of features except for the leukemia Golub dataset where f = 0.7 was better. Best results are shown in bold. value from an f-test. This forward-selection strategy is similar to that in PAM (Tibshirani et al., 2002) and the Prophet tool in GEPAS (Montaner et al., 2006).
We first compared our algorithm with the sbPSO. For the uuPSO runs, we used an overall controlling proxy parameter, size =10, then b = size, u = size, e = 1, n = 0.5*size, and m = 1.5*size. The results summarized in Table 2 are encouraging. Our algorithm increased remarkably the classification accuracy in all datasets, decreased the selected features from thousands to a handful, and decreased the computation time from days to about an hour. Best model accuracy improved from around 7.3% in Leukemia-Golub to 18.1% in BC. Moreover, remarkable differences are observed in model size and computation time where our algorithm gets better results by around two orders of magnitude. Overall, these results clearly show that our algorithm is able to obtain better predictive models from functional genomics data with thousands of features in a fraction of time.
We first compared our algorithm with the sbPSO. For the uuPSO runs, we used an overall controlling proxy parameter, size =10, then b = size, u = size, e = 1, n = 0.5*size, and m = 1.5*size. The results summarized in Table 2 are encouraging. Our algorithm increased remarkably the classification accuracy in all datasets, decreased the selected features from thousands to a handful, and decreased the computation time from days to about an hour. Best model accuracy improved from around 7.3% in Leukemia-Golub to 18.1% in BC. Moreover, remarkable differences are observed in model size and computation time where our algorithm gets better results by around two orders of magnitude. Overall, these results clearly show that our algorithm is able to obtain better predictive models from functional genomics data with thousands of features in a fraction of time.
Theory and New Applications of Swarm Intelligence Fig. 1. Representative genes selected by our algorithm from 100 runs for the BC dataset. (A) A heatmap representation of the most frequent genes. Darker (red) or lighter (green) colors represent higher or lower expression respectively. Ord represents the order of genes given by frequency in 100 models (shown in parenthesis). (B) Class centroid for each gene. Negative values represent under-expression relative to a common reference.
Theory and New Applications of Swarm Intelligence Fig. 1. Representative genes selected by our algorithm from 100 runs for the BC dataset. (A) A heatmap representation of the most frequent genes. Darker (red) or lighter (green) colors represent higher or lower expression respectively. Ord represents the order of genes given by frequency in 100 models (shown in parenthesis). (B) Class centroid for each gene. Negative values represent under-expression relative to a common reference.
to increase from the initial value of size. The smallest average accuracy was 95% for size=5 but tending for model sizes around 8, which was limited by the m parameter used. This suggests that the decrease in accuracy is due to the low number of features used. This is sensible because the number of classes for LY dataset is 5, so, the use of 8 features is a compromise between high accuracy at low number of features. Overall, these results show that our implementation is able to control the number of active features and support our proposal that a high number of features for accurate classification of functional genomics data is not necessary.
to increase from the initial value of size. The smallest average accuracy was 95% for size=5 but tending for model sizes around 8, which was limited by the m parameter used. This suggests that the decrease in accuracy is due to the low number of features used. This is sensible because the number of classes for LY dataset is 5, so, the use of 8 features is a compromise between high accuracy at low number of features. Overall, these results show that our implementation is able to control the number of active features and support our proposal that a high number of features for accurate classification of functional genomics data is not necessary.
Fig. 3. Initialization tests for the BC (A-B) and LY (C-D) datasets. (A-C) Accuracy evolution. (B-D) Dynamics of the number of active features. Points as in Figure 2. The runs presented above, which changed all parameters coordinately, show the overall behavior for different settings. However, the effects of each adaptation are not well appreciated. Therefore, we tested each adaptation separately. To challenge the initialization adaptation, we varied b from 5 to 400 setting b={5,10,25,50,100,400} while other parameters were kept fixed (u=10, e=1, n=2, m=600). Results are shown in Figure 3. For the BC dataset (Figure 3 A-B), accuracy was clearly smaller for b=400 within the first 50 epochs. On the other hand, for the LY dataset the accuracy was higher for larger values of b (Figure 3 C-D). For both BC and LY datasets, the final number of active features decreased when b decreased. Such decrease would be beneficial for our purposes since potential biomarker
Fig. 3. Initialization tests for the BC (A-B) and LY (C-D) datasets. (A-C) Accuracy evolution. (B-D) Dynamics of the number of active features. Points as in Figure 2. The runs presented above, which changed all parameters coordinately, show the overall behavior for different settings. However, the effects of each adaptation are not well appreciated. Therefore, we tested each adaptation separately. To challenge the initialization adaptation, we varied b from 5 to 400 setting b={5,10,25,50,100,400} while other parameters were kept fixed (u=10, e=1, n=2, m=600). Results are shown in Figure 3. For the BC dataset (Figure 3 A-B), accuracy was clearly smaller for b=400 within the first 50 epochs. On the other hand, for the LY dataset the accuracy was higher for larger values of b (Figure 3 C-D). For both BC and LY datasets, the final number of active features decreased when b decreased. Such decrease would be beneficial for our purposes since potential biomarker
models would become more compact. Nevertheless, when initialization was smaller than update (b =5) for the BC dataset, the number of active features increases. This increase was also observed in LY for b <= 20. For the BC dataset a trend for around 10 active features is clearly observed even for b =400. In this case, it took 600 epochs to decrease the number of active features from 400 to less than 20 and to reach the same level of accuracy than in the other runs. The decrease in the number of features and the poor accuracy for runs with high b suggest that the number of active features is excessive. This may happen when the classifier is overloaded with noisy features. On the other hand, for the LY dataset, the number of active features does not decrease systematically such as that in the BC dataset. Presumably due to high accuracies in LY that are already close to 1 (the maximum possible value); thus no major evolutive pressure is present. Overall, runs tend to adjust the number of active features depending on the starting point and the accuracy pressure.
models would become more compact. Nevertheless, when initialization was smaller than update (b =5) for the BC dataset, the number of active features increases. This increase was also observed in LY for b <= 20. For the BC dataset a trend for around 10 active features is clearly observed even for b =400. In this case, it took 600 epochs to decrease the number of active features from 400 to less than 20 and to reach the same level of accuracy than in the other runs. The decrease in the number of features and the poor accuracy for runs with high b suggest that the number of active features is excessive. This may happen when the classifier is overloaded with noisy features. On the other hand, for the LY dataset, the number of active features does not decrease systematically such as that in the BC dataset. Presumably due to high accuracies in LY that are already close to 1 (the maximum possible value); thus no major evolutive pressure is present. Overall, runs tend to adjust the number of active features depending on the starting point and the accuracy pressure.
Under-Updated Particle Swarm Optimization for Small Feature Selection Subsets from Large-Scale Dataset
Under-Updated Particle Swarm Optimization for Small Feature Selection Subsets from Large-Scale Dataset
and 11 active features respectively. This tendency toward lower values indicates that the optimal number of active features needed could be lower than 9. Considering these observations, there is a tendency for the number of active features closer to 6. A consequence of this result is that the shape of this distribution may suggest an estimation of the optimal value of the number of active features.
and 11 active features respectively. This tendency toward lower values indicates that the optimal number of active features needed could be lower than 9. Considering these observations, there is a tendency for the number of active features closer to 6. A consequence of this result is that the shape of this distribution may suggest an estimation of the optimal value of the number of active features.
Table 4. Distribution of the number of active features, from 5 to 15, in runs at different values of n and m for the BC dataset. The highest frequency is shown in bold.
Table 4. Distribution of the number of active features, from 5 to 15, in runs at different values of n and m for the BC dataset. The highest frequency is shown in bold.
Fig. 7. (A) Top genes found in 100,000 runs for the BC dataset. Inset: Comparison of feature selection frequency for 10,000 and 90,000 runs. (B) Generation of the negative dataset and positive added features. Features were sorted by the number of times they were found in the best particle for 100,000 runs. Top 494 were removed to generate the negative dataset. Top features were then injected progressively to the negative dataset: first 10, then 20, 50, 100, 200, 300, 400 and 494 (from leftmost dashed line to rightmost dashed line). We consider that this value is very conservative since we used raw p-values uncorrected for multiple tests. The number of features whose frequency surpasses this value was 494. Consequently the negative datasets carried 2,426 features. The number of 100,000 was chosen in order to be confident that the rank of top genes would not change among different runs. Indeed ranks between two runs of 10,000 and 90,000 have no major differences (see Figure 7A inset). If our algorithm cannot find related features in this negative dataset, a decrease in the final accuracy would be expected. As projected, the mean accuracy in 1000 runs for the negative dataset was 72% (labeled with BOTTOM in Figure 7B) while the same indicator in the original dataset was 82% (labeled with FULL in Figure 8B). Then, we inserted only the original top 10 features to the negative dataset (representing 0.4% of the 2,436 total features) and counted how many times the same top 10 features were found in the best models for 1000 runs. We also performed this procedure for top 20, 50, 100, 200, 300, 400 and 494 originally removed features. Results are shown in Figure 8A. We found that the high accuracy is easily restored after the first insertion (labeled TOP10 in Figure 8B). However, results show that the next 10 features were needed to fully restore the original observed accuracy distribution (labeled TOP20 in Figure 8B). We found all 10 and 20 top inserted features and more than 90% among the top 100 injected features. These results indicate that most frequent features (and presumably those more related to classes) are easily found even in highly noisy scenarios (99.6% of noisy features). Nevertheless, search efficiency was decreased to 60% when 494 top features were added. Within this, top 50 features were always found. Therefore, the search capability was uncertain only for those features ranked in the order of hundreds, presumably, because of random effects.
Fig. 7. (A) Top genes found in 100,000 runs for the BC dataset. Inset: Comparison of feature selection frequency for 10,000 and 90,000 runs. (B) Generation of the negative dataset and positive added features. Features were sorted by the number of times they were found in the best particle for 100,000 runs. Top 494 were removed to generate the negative dataset. Top features were then injected progressively to the negative dataset: first 10, then 20, 50, 100, 200, 300, 400 and 494 (from leftmost dashed line to rightmost dashed line). We consider that this value is very conservative since we used raw p-values uncorrected for multiple tests. The number of features whose frequency surpasses this value was 494. Consequently the negative datasets carried 2,426 features. The number of 100,000 was chosen in order to be confident that the rank of top genes would not change among different runs. Indeed ranks between two runs of 10,000 and 90,000 have no major differences (see Figure 7A inset). If our algorithm cannot find related features in this negative dataset, a decrease in the final accuracy would be expected. As projected, the mean accuracy in 1000 runs for the negative dataset was 72% (labeled with BOTTOM in Figure 7B) while the same indicator in the original dataset was 82% (labeled with FULL in Figure 8B). Then, we inserted only the original top 10 features to the negative dataset (representing 0.4% of the 2,436 total features) and counted how many times the same top 10 features were found in the best models for 1000 runs. We also performed this procedure for top 20, 50, 100, 200, 300, 400 and 494 originally removed features. Results are shown in Figure 8A. We found that the high accuracy is easily restored after the first insertion (labeled TOP10 in Figure 8B). However, results show that the next 10 features were needed to fully restore the original observed accuracy distribution (labeled TOP20 in Figure 8B). We found all 10 and 20 top inserted features and more than 90% among the top 100 injected features. These results indicate that most frequent features (and presumably those more related to classes) are easily found even in highly noisy scenarios (99.6% of noisy features). Nevertheless, search efficiency was decreased to 60% when 494 top features were added. Within this, top 50 features were always found. Therefore, the search capability was uncertain only for those features ranked in the order of hundreds, presumably, because of random effects.
Fig. 8. (A) Search efficiency in the top added features. (B) Fitness distribution for uuPSC search capability tests. We then asked about the amount of information that uuPSO is able to find in the original dataset. That is, the improvement attributed to the uuPSO. For this, we generated random models of equal sizes than those observed in the models generated for the negative dataset. These models would represent the amount of information (given by accuracy) found by random chance. Similarly, we generated series of 20 random models recording only the best in each series. These would represent the best models that drive the search at the beginning of the swarm optimization procedure. Results show that the baseline for the amount of information is about 54% given by random models in the negative dataset (labeled with Random in Figure 9B). The accuracy distribution from the best of the random series indicates that uuPSO starts with 66% accuracy (labeled with Best Random in Figure 8B), which is then improved to 72% during the uuPSO process (labeled with BOTTOM in Figure 8B). Overall, these results show that the average improvement by uuPSO in the BC dataset is 16% (from 66% to 82%) when good predictors are present.
Fig. 8. (A) Search efficiency in the top added features. (B) Fitness distribution for uuPSC search capability tests. We then asked about the amount of information that uuPSO is able to find in the original dataset. That is, the improvement attributed to the uuPSO. For this, we generated random models of equal sizes than those observed in the models generated for the negative dataset. These models would represent the amount of information (given by accuracy) found by random chance. Similarly, we generated series of 20 random models recording only the best in each series. These would represent the best models that drive the search at the beginning of the swarm optimization procedure. Results show that the baseline for the amount of information is about 54% given by random models in the negative dataset (labeled with Random in Figure 9B). The accuracy distribution from the best of the random series indicates that uuPSO starts with 66% accuracy (labeled with Best Random in Figure 8B), which is then improved to 72% during the uuPSO process (labeled with BOTTOM in Figure 8B). Overall, these results show that the average improvement by uuPSO in the BC dataset is 16% (from 66% to 82%) when good predictors are present.
Table 6. Comparison of uuPSO algorithm using the nearest centroid (NC) and support vector machine (SVM) classifiers. The goodness (Accuracy) and number of features from the best model found among 100 runs is shown. Goodnesses and features mean and standard deviation (SD) were estimated along 1000 epochs and 100 runs. Time was estimated for 100 runs. A star (*) marks times estimated from partial results. generally smaller. These differences are consistent with the perception that SVM is a powerful classifier. However, we note a huge difference in processing time, which should correspond to the complex training procedure of SVM, to the java implementation used, or to our coupling implementation of this library. In the Ovarian dataset, uuPSO using SVM found models with 100% accuracy in early epochs explaining the small difference in processing time for this dataset.
Table 6. Comparison of uuPSO algorithm using the nearest centroid (NC) and support vector machine (SVM) classifiers. The goodness (Accuracy) and number of features from the best model found among 100 runs is shown. Goodnesses and features mean and standard deviation (SD) were estimated along 1000 epochs and 100 runs. Time was estimated for 100 runs. A star (*) marks times estimated from partial results. generally smaller. These differences are consistent with the perception that SVM is a powerful classifier. However, we note a huge difference in processing time, which should correspond to the complex training procedure of SVM, to the java implementation used, or to our coupling implementation of this library. In the Ovarian dataset, uuPSO using SVM found models with 100% accuracy in early epochs explaining the small difference in processing time for this dataset.
Note: NLSE is the least squared error divided by the standard deviation of Y, and RSQ is the R-Square statistic from the regression. An NLSE of 0.0 is perfect while an RSQ of 1.0 is perfect. Table 1. Sample Test Problem Regressions Clearly the AEG symbolic regression runs are discovering and optimizing the embedded constants correctly; however, the standard GP symbolic regression runs are unable to optimize the constants and get confused. It is simply too difficult for standard GP to optimize these difficult embedded constants using only mutation and crossover. Furthermore, the standard GP runs produce estimators which are far from the correct form. The following are the top five estimators, produced by the standard GP symbolic regression, for test problem (14.1).
Note: NLSE is the least squared error divided by the standard deviation of Y, and RSQ is the R-Square statistic from the regression. An NLSE of 0.0 is perfect while an RSQ of 1.0 is perfect. Table 1. Sample Test Problem Regressions Clearly the AEG symbolic regression runs are discovering and optimizing the embedded constants correctly; however, the standard GP symbolic regression runs are unable to optimize the constants and get confused. It is simply too difficult for standard GP to optimize these difficult embedded constants using only mutation and crossover. Furthermore, the standard GP runs produce estimators which are far from the correct form. The following are the top five estimators, produced by the standard GP symbolic regression, for test problem (14.1).
Note: Per annum compound returns for each decade. Theory and New Applications of Swarm Intelligence 13. Buying current earnings yield
Note: Per annum compound returns for each decade. Theory and New Applications of Swarm Intelligence 13. Buying current earnings yield
The data supports the conclusion that even using this current revenue proxy model buying the top five, twenty five, and fifty stocks with the highest (current12MoEPS/currentPrice) produces above average securities investing profits, as least for the 1500 Valueline stocks, as shown in Table 3.
The data supports the conclusion that even using this current revenue proxy model buying the top five, twenty five, and fifty stocks with the highest (current12MoEPS/currentPrice) produces above average securities investing profits, as least for the 1500 Valueline stocks, as shown in Table 3.
Predicting Corporate Forward 2 Month Earnings
Predicting Corporate Forward 2 Month Earnings
The per annum returns using this symbolic regression revenue prediction model buying the top five , twenty five, and fifty stocks with the highest (regression12MoEPS/currentPrice, produces above average securities investing profits as shown in Table 7.
The per annum returns using this symbolic regression revenue prediction model buying the top five , twenty five, and fifty stocks with the highest (regression12MoEPS/currentPrice, produces above average securities investing profits as shown in Table 7.
Related topics:
Computer ScienceOperations ResearchSwarm IntelligenceBio Inspired Computing
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved