An Improved Particle Swarm Optimization for Prediction Model of Macromolecular Structure Fuli RONG, Yang YI，Yang HU Information S cience S chool & Technology, Sun Yat-sen University GuangZhou, 510275, China E-mail:[email protected]

Abstract A novel particle swarm optimization combined with filter method (FM _PSO) is proposed in this paper. FM _PSO adopts the filter technique which can broaden the iteration acceptance criterion to improve the exploitation ability in the initial stage. Simultaneously, the divide and conquer method is considered to speed the convergence rate. FM _PSO can improve the ability of exploration and exploitation, despite of large scale optimization. Numerical tests show that FM_PSO is feasible and efficient both in the test function and prediction model of macromolecular structure, and outperforms other algorithms. Key words: PSO, Filter Method, Macromolecular Structure Prediction

1. Introduction Molecular modeling gives rise to a wide variety of global optimiza tion problems according to different models derived fro m chemistry, physics, biology, etc, wh ich plays an important part in discovering new material and producing new medicine. The main types of the macro molecular model include potential energy model, distance geometry model, emp irical force field model, etc, where potential energy minimizat ion received most attention. Owing to its properties of mult i-extreme and mu lti-dimensional, it turns out to be difficu lt for global optimization. The main methods applied in this model include random method [1], branch bound method [2], genetic algorith m [3], simulated annealing algorith m [4] etc. In order to solve the burden of cost time and shortages in large scale situation, this paper proposes an improved particle swarm optimization comb ining with filter technique and divide and conquer strategy, efficient for large scale optimization. The particle swarm optimization (PSO) originally proposed by Kennedy and Eberhart as a simulat ion of social behavior has developed greatly in recent years, widely used in many areas such as image process [5], job shop scheduling [6], dynamic resource allocation [7], etc. In view of the shortages of basic PSO, Sh i and Eberhart [8] introduced an inert ia weight to controls the impact of previous velocity of particle on its current one. Later, they adjusted it linear decrease with the generation [9], and

proposed a maximal speed vmax controlled the exp loring ability. Then Jiang [10] divided the entire space into mu ltip le sub-swarms, each of which is made to evolve based on PSO algorith m. Hu [11] proposed the improved PSO based on the simp le evolutionary equations and the extremu m disturbed arith metic operators to overcome the demerits of basic PSO. He [12] used a special mutation named escape operator to make particles explo re the search space more efficiently. Despite the above-mentioned methods have imp roved the performance of PSO, there are some shortages in solving the high-dimensional optimization problems and premature convergence. Inspired by the filter method we propose an improved particle swarm optimization (FM_PSO) in this paper. In FM_PSO, filter technique is adopted to broaden the iteration acceptance criterion, such that FM_PSO can improve the exp lo itation ability in the in itial stage. Simu ltaneously, the divide and conquer method is considered to partition the entire swarm into several sub-swarms, each of which is made to evolve based on FM_PSO, then merge together to speed the convergence rate. FM_PSO can avoid premature convergence effectively and run more efficiently especially for large scale optimization.

2. Overvie w of standard PSO The PSO is init ialized with a population of random particles, each particle ad justs itself by tracking two extreme called ind ividual best particle x_pbest and global best particle x_gbest for each generation. Suppose that the search space is n-dimensional, and then the particle i of the swarm can be represented by an n-dimensional vector Xi =(xi1 ,xi2 ,…,xin ), its velocity can be represented by Vi =(vi1 ,vi2 ,…,vin ), the other informat ion of particle can be denoted similarly. At each step, the velocity and position of particle will be updated according to the following two equations: vid (k+1)＝w*vid (k)+c1 *r1*[x_pbest id (k)－xid (k)]＋c2* r2 *[x_gbestgd (k)－xid (k)] (1) xid (k+1)= xid (k)+ vid (k+1) (2)

where, the inertia weight w updates as follo ws: w=(w1 -w0 )*(max_gen-gen)n /max_genn + w0 (3) w0 denotes initial inert ia weight, w1 denotes final inertia weight, max_gen denotes the maximu m generations, gen represents the current generation. In Eq. (1), cl and c2 are positive constant parameters called acceleration coefficients. r1 and r2 are random variables independently uniformly distributed with range (0,1).

3. The theory and description of FM_PSO 3.1. Filter method theory Filter method was first introduced by Fletcher and Leyffer [13] as a way to globalize SLP (sequential linear programming) and SQP (sequential quadratic programming). Filter methods are designed to solve nonlinear programming problem:  min x¡ n f ( x)    s.t.c( x)  0

(4)

where we assume f : R→ R and c : R→ R are t wice continuously differentiable. There are two co mpeting aims to measure whether to accept a new iteration. The first is the minimizat ion of the objective function and the second is the satisfaction of the constraints. Conceptually, these two conflicting aims can be written as follow: minimize f ( x)  minimize h(c( x))

(5)

m

m

j 1

j 1

where h(c(x))= h(c( x))   c j ( x)   max(0, c j ) is the l1 norm o f the constraint violation. In this paper, we modify the filter function inspired by [14] to control the searching space adaptively as follow. 0if u ( x)  0 Definiti on 1. Filter function. h( x)   u ( x)else

(6)

where u(x)=f(x)-min{f(x_lbest), f(xi ) | xi ∈ N(x)}, N (x) denotes the neighborhood of x. There are many topological structures in PSO, this paper considers the ring structure. Definiti on 2. Do mination. A pair (f(x(k)), h(x(k))) obtained on iteration k is said to dominate another pair (f(x(l ), h(x(l))) if and only if both f(x(k)) < f(x(l)) and h(x(k))< h(x(l)). Definiti on 3. Filter. A filter is a list of pairs(f(x),h(x)) such that no pair dominates each other. Definiti on 4. Acceptance criterion. Letting F denotes the filter, and (f(x(i)),h(x(i))), i=1,2,…,n are the elements in the filter. x can be accepted by filter F(k) if for all xi ∈F(k) it satisfies h(x)<(1-η)h(xi ) or f(x)
Definiti on 5. Filter update. If the current iteration x(k) is accepted by filter, then include it in the filter and remove all the points dominated by x(k).

3.2. FM_PSO The follo wing two heuristic guidelines used to improve the PSO performance. Rule 1. A novel opti mal update criterion. Co mbin ing with the filter technique, this rule modifies the conventional simple decrease iteration acceptance criterion as the filter technique pattern, wh ich can broaden the iteration searching range to obtain global optimal with high probability, especially for h igh dimensional situation. If the current x_gbest can be accepted by the filter, x_gbest can be included in the filter, and the filter is updated for the next generation. Its acceptance criterion are as follows. The conventional update criterion for x_gbest: min f ( x(k )) if min f ( x(k ))  x _ gbest x _ gbest    x _ gbest other

(8)

The novel update criterion for x_gbest: min f ( x(k )) if min f ( x(k )) is accepted by filter x _ gbest    x _ gbest other

(9) whether min f(x(k)) can be accepted by filter can reference to definition 4. Theorem 1. If f(x) is limitary, the solution will be obtained from Ru le 1 is a local optimal. Proof: During the updation process of the solution, if and only if the following conditions satisfy, that is f(x_gbest(k+1))
f (x_gbest (k + N ))
Let N→∞，because f(x) is limitary, N

 h(x_gbest (k +j )) <+∞ j 1

Then

lim h(x_gbest (k +j ))=0 . j 

If all the iterates are chosen by satisfying Eq. (11)， then we have h(x_gbest (k + N ))<(1- ) N h(x_gbest (k )) Let N→∞, because (1-η)<1, lim h(x_gbest (k +j ))=0 then j  In the actual process, situation Eq. (10) and Eq. (11) will occur in turn, lim h(x_gbest (k +j ))=0 Thus j  According to the definition of filter function in definit ion 1, when N→∞, f(x_gbest(N)) - min{f(x_lbest(N)), f(xi ) | xi ∈N(x)}≤ 0,

thus x_gbest(N) is a local optimal. Rule 2 Di vi de and conquer method. This ru le div ides the entire swarm into 2k sub-swarms, each of wh ich will do the evolution computation based on particle swarm optimization. As the generation grows, every two sub-swarms will merge into one until all sub-swarms being a whole space. The mergence of sub-swarm occurs when the generation equals the multiple of max_gen/(K+1), and then the number of sub-swarms will reduce to the value field of (2k to 2k-1 ), where  max_ gen  gen  )( K  1)  , and 2K is the in itial nu mber of max_ gen  

k= (

sub-swarm, that requires the population size N=2K. Every two sub-swarms merge into one and the better global best individual is chosen as the x_gbest of new sub-swarm, and the filter of new sub-swarm should select according to the filter updation rule. In the process of sub-swarm mergence, the information can be shared between each sub-swarm. Property 1. In the initial stage of PSO, the exploring ability should be focused to avoid premature convergence, while the convergence rate should be focused in the final stage. In the init ial stage, the whole swarm is div ided into several sub-swarms, and the best individual in the sub-swarm is updated according to the filter technique, which can broaden the simple decrease acceptance criterion to improve the exp loitation ability and avoid premature convergence. While in the final searching stage, we merge the sub-swarm to make it gathering to the best solution, and the two aims f(x) and h(x) in the filter are decreasing with the iteration increasing, so its acceptance range is shrinking, which is good for the swarm gathering to the best individual, despite of high dimensional problem, its convergence rate can be also guaranteed. The algorith m is described as follows. Step 1 Create N random individuals as swarm G, and partition it into P sub-swarms, in itial the velocity V for each individual. Step 2 Set individual best position to the current position for each particle, and co mpute the global position xj_gbest for sub-swarm Ωj Step 3 while a stopping criterion is not met: 3.1 Update the velocity and position for each particle according to Eq. (1) and Eq. (2). If xi < Xmin, let xi = Xmin, if xi >Xmax, let xi =Xmax. If vi < Vmin, then vi = Vmin, else if vi >Vmax, then vi =Vmax. 3.2 Find the fitness value for each particle, if f(xi )
Step 4 Co mpute the best value in filter F1 (k) as the global optimal.

4. Numerical experiments In this section, FM_PSO is tested on some famous functions, and then it is applied to the prediction model of macro molecular structure. Set population size N=64, acceleration coefficients cl =c2 =2.0, w ranging fro m 0.4 to 0.9, and the parameters η and γ used in filter function are 0.1. All the tests are run on a computer with Intel(R) Pentium(R) M2.66G, 512M of memo ry.

4.1. Mathematical functions In this part, we verify the feasibility and efficiency of FM_PSO by testing following functions . TF.1 Extended Powell n/4

f ( x)   ( x4i 3  10 x4i  2 )2  5( x4i 1  x4i )2  ( x4i  2  2 x4i 1 )4 i 1

10( x4i 3  x4i )4 9

2 2 2 TF.2 Dixson f ( x)  (1  x1 )  (1  x10 )   ( xi  xi 1 ) i 1

TF.3 Trigonometric n

n

i 1

j 1

f ( x)   (n   cos x j  i (1  cos xi )  sin xi ) 2 n

TF.4 Mancino f ( x)   ( i 1

n



j 1, j  i

1

[( x 2j  i / j ) 2

1 2

1

(sin 5 log( x 2j  i / j )  cos5 log( x 2j  i / j ) 2 )]  14nxi  (i  n / 2)3 ) 2 n/2

2 2 2 TF.5 Rosenbrock f ( x)  100( x2i  x2i 1 )  (1  x2i 1 ) i 1

n

2 2 2 TF.6 Nondia f ( x)  100( x1  xi )  (1  xi ) i 2

n

2 TF.7 Trid ia f ( x)   i(2 xi  xi 1 ) i 2

TF.8 Variably Dimensioned n

n

n

i 1

i 1

i 1

f ( x)   ( xi  1)2  [ i( xi  1)]2  [ i( xi  1)]4

The numerical results are shown in table 1.In the table, n represents the dimension of the test function, SPSO represents Standard PSO. All the theoretical global optimu ms of this function are 0. Each simu lation is implemented for 100 rounds to calculate the cu mulat ive frequency of success times, of wh ich accuracy is less than 10-5 . Fro m the table, FM_PSO can ach ieve the optimal for all test functions, and has better results in the cu mulat ive frequency. When the generation is less that 10000, it can’t achieve a h igh accuracy optimal efficiency for TF5 and TF6, however, with the generation increasing, satisfied results can be obtained. In brief, FM_PSO can solve the

test function effectively and perform better than standard PSO. Table 1. Results of FM_PSO in test functions

No.

n

Generations

Optimal of FM _PSO

TF.1 TF.2 TF.3 TF.4 TF.5

4 10 10 4 10 10 4 4 10 10

10000 10000 10000 10000 10000 50000 10000 50000 10000 10000

1.780×10-10 3.487×10-20 1.441×10-13 3.155×10-30 3.140×10-4 1.219×10-17 1.269×10-5 6.397×10-7 1.155×10-33 0

TF.6 TF.7 TF.8

Success times of SPSO 100 93 100 100 0 47 0 55 97 100

Success times of FM _PSO 100 100 100 100 0 71 1 63 100 100

4.2. Application of FM_PSO in Prediction Model of Macromolecular Structure 4.2.1. Prediction model of macromolecular structure. Experience potential energy function in physics called mo lecular field is a potential energy function which is suitable to computing macro molecular structure according to the classical mechanical model and many experimental spectra data. Generally, potential energy mo lecular modeling in use today can be interpreted in terms of a relatively simple 4-co mponent picture of the intra and intermo lecular forces. So me mo re sophisticated models may have other additional terms but inevitably contain these four components: E＝E1＋E2＋E3＋E4 (12) E1   cij1 (lij  lij0 )2 E2   cij2 (ij  ij0 )2 ( i , j )M1 ( i, j) M 2 Where , , E3 



( i , j )M 3

cij3 (1  cos(3ij  ij0 ))

E4 

(-1) i ( i , j ) M 3 ri j



, E denotes the potential energy which is a function of the positions of M particles. The first term represents the

interaction between pairs of atoms modeled by a potential which gives the increase in energy as the bond length l ij deviates from the reference value l0 ij . The second term is a summation over all valence angles in the molecular. The third term is a torsional potential that models how the energy changes as a bond rotates. The fourth contribution is the non-bond term usually using potential term. Unfortunately, the model given above is still too difficult for mathematical co mputation. So, it needs further predigestion and refinement. Among all the four terms, E3 and E4 have more significant impact on mo lecular construction, which means E can be simplified as follows when the bond lengths and bond angles are assumed to be fixed at their equilibriu m values (i.e. E1 and E2 are equivalent 0). Thus, the objective function is: E= E3 +E4 =



cij2 (1  cos(3wij  wij0 ))  

torsions

i, j

BB Optimal

10 16 20 28 50 200 400 600 1000 1500 2000

-0.5894 -0.8361 -1.0006 -1.3295 / / / / / / /

SPSO Optimal -0.00287892 -0.00460627 -0.00575784 -0.00806093 -0.0143946 -0.0575784 0.116216 6.92703 61.3146 188.95 437.616

FM _PSO Optimal -0.00287892 -0.00460627 -0.00575784 -0.00806093 -0.0143946 -0.0575784 -0.109774 0.329442 9.99385 168.5 285.75

(13)

4.2.2. Numerical Test on Macromolecular Structure Model. Table 2 is the nu merical results for solving Eq. (13) model using FM_PSO, and it’s co mpared with the results calculated by branch-and-bound (BB) method [2] and SPSO. n represents the dimension. The parameters used in SPSO and FM_PSO are uniformity, and their generation for different dimension are all 10000. Fro m the Table 2, we can draw the conclusion easily that the proposed method is not only feasible but also efficient with fo llowing advantages: (1)Increasing the computational accuracy Co mpared with results obtained by BB method, the computational accuracy of FM_PSO can reach 10-3 , wh ile the BB reaches 10-1 at best. With the dimension increasing, the computational accuracy o f FM_PSO is h igher than SPSO. (2) Increasing the computational effectiveness With the problem scale increasing, the co mputational t ime of BB algorith m is increasing with exponential growth. When it computes the 28 dimensions situation, it takes 82.528 hours. FM_PSO only spends about 600 seconds for the calculation of 600 dimensions, which also costs less time than standard PSO. Due to the number of iterat ions is

Table 2. Numerical results for solving Eq. (12) model using FM_PSO

n

(1)i ri j

BB CPU time (Second) 2.47 50.53 1167 297100 / / / / / / /

SPSO CPU time（Second） 11.89 19.015 23.609 33.078 58.14 232.187 465.235 697.079 1169.72 1766.48 2383.75

FM _PSO CPU time（Second） 10.938 16.328 22.469 30.266 48.907 193.281 399.843 581.546 994.297 1524.17 2070.17

not enough when the dimension is larger than 1000, its accuracy can not control under 10-1 , but compared to the SPSO, FM_PSO can convergence faster, its calculat ion time still can be controlled with the increases of problem scale. (3) Excellent performance on the co mputation of higher dimension problems The BB algorith m beco mes infeasible when n> 28, while the proposed algorith m does very well when n = 2000 by increasing the generation. All in all, great imp rovements have been made by the proposed method.

5. Conclusions This paper proposed an improved PSO co mbin ing with filter technique and divide and conquer method. The main contribution can conclude as follow: (1) Modify the iteration acceptance criterion by filter technique to broaden the iteration searching range, wh ich can avoid premature convergence and improve the exploring ability. (2) Adopt the divide and conquer method to partit ion the swarm into several sub-swarms and combine them with the generation increasing, wh ich can accelerate the convergence rate and improve the exploitation ability. (3) Apply the FM_PSO in the predict ion model of macro molecular structure, and it can solve high dimensional situation that other algorith m can not achieve. Nu merical tests show its feasibility and effectiveness both in test functions and prediction model of macro mo lecular structure. However, FM_PSO is not sufficient in the global convergence analysis, the further research should focus on its theory and application in large scale problem. Acknowledg ments This paper is supported by the National Natural Science Foundation of China under grant No. 60573159.

Fig. 1 Comparison of FM_PSO and SPSO in solving 1000 dimensions

References

Fig. 2 Comparison of FM_PSO and SPSO in solving 600 dimensions

Fro m the Figure 1, FM_PSO declines slower than SPSO in the initial stage, but it can convergence faster and gain the optimal with high accuracy with the generation increasing. Figure 2 only describes the status after 6000 generations, but it also demonstrates that FM_PSO can convergence to the optimal faster with high accuracy.

[1] Z. Zou, R.H. Byrd, R.B. Schnabel, ―A Stochastic/ Perturbation Global Optimization Algorithm for Distance Geometry Problems‖, Journal of Global Optimization, Vol.11, No.1, 1997, pp. 91-105. [2] C. Lavor, N. M aculan, ―A Function to Test M ethods Applied to Global M inimization of Potential Energy of M olecules‖, Numerical Algorithms, Vol.35, 2004, pp. 287-300. [3] D.M . Deaven, N. Tit, J.R. Moorris, K.M. Ho, ―Structural Optimization of Lennard-jones Clusters by A Genetic Algorithm‖, Chemical physics letters, Vol.256, No.2, 1996, pp. 195-200. [4] J. Zhao, H.W. Tang, ―An Improved Simulated Annealing Algorithm and Its Application‖, Journal of Dalian University of Technology, Vol.46, No.5, 2006, pp. 775-780. [5] M .P. Wachowiak, R. Smolíková, et al, ―An Approach to M ultimodal Biomedical Image Registration Utilizing Particle Swarm Optimization‖, IEEE Transactions on Evolutionary Computation, Vol.8, No.3, 2004, pp. 289-301. [6] J.L. Ching, T.T. Chao, L. Pin, ―A Discrete Version of Particle Swarm Optimization for Flowshop Scheduling Problems‖, Computers & Operations Research, Vol.34, No.10, 2007,pp. 3099-3111. [7] D. Vengerov, ―A Reinforcement Learning Approach to Dynamic Resource Allocation‖, Engineering application of artificial intelligent, Vol.20, No.3, 2007, pp. 383-390. [8] Y. Shi, R.C. Eberhart, ―A M odified Particle Swarm Optimizer‖, in IEEE International Conference on Evolutionary Computation, Piscataway, NJ, 1998, pp. 69~73.

[9] Y. Shi, R.C. Eberhart, ―Parameter Selection in Particle Swarm Optimization‖, in 7th International Conference on Evolutionary Programming VII, California, USA, 1998, pp. 591—600. [10] Y. Jiang, T. Hu, et al, ―An Improved Particle Swarm Optimization Algorithm‖, Applied M athematics and Computation, Vol.193, No.1, 2007, pp. 231-239. [11] W. Hu, Z.S. Li, ―A Simpler and M ore Effective Particle Swarm Optimization Algorithm‖, Journal of Software, Vol.18, No.4, 2007, pp. 861-868. [12] R. He, Y.J. Wang etc, ―An Improved Particle Swarm Optimization Based on Self-Adaptive Escape Velocity‖, Journal of Software, Vol.16, No.12, 2005, pp. 2036-2044. [13] R. Fletcher, S. Leyffer, ―Nonlinear Programming without A Penalty Function‖, M athematical Programming, Vol.91, No.12, 2002, pp. 239–269. [14] T. Wu, L. Sun, ―A Filter-Based Pattern Search M ethod for Unconstrained Optimization‖, Numerical M athematics A Journal of Chinese Universities English Series, Vol.15, No. 3, 2006, pp. 209-216.