ADM2006 - Paper No.309 Proceedings of the International Conference on Advanced Design and Manufacture 8-10 January, 2006, Harbin, China

A NOVEL EVOLUTIONARY ALGORITHMS BASED ON NUMBER THEORETIC NET FOR NONLINEAR OPTIMIZATION Fei Gao Dep. of Mathematics, Wuhan University of Technology, 430070, P. R .China E-mail: [email protected]

Abstract: How to detect global optimums which reside on

optimize the problems such as genetic algorithms,

complex function is an important problem in diverse scientific

evolutionary algorithms, simulated annealing and etc.

fields. Deterministic optimization strategies, such as the

Stochastic methods have been proved to be capable

Newton–family algorithms, have been widely applied for the

of finding global optimum by an asymptotic convergence.

detection of global optimums. However, in the case of

These ideas require a primary stochastic evolving form

discontinuous /non-differentiable /non-convex complex functions,

and converge to one optimum. The samples generated by

this approach is not valid. In such cases, stochastic optimization

evolving operators such as selection, mutation and

strategies simulating evolution process have proved to be a

acceptance concentrate around from a local optimum to

valuable tool. In this paper, a novel evolutionary algorithm

global optimum in terms of computational efforts needs.

based on Number Theoretic Net for detecting global optimums of

This problem may be solved by improving the

Complex functions is introduced. It can be applied to any

distribution of points so as to mine the information of

functions with multiple local optimums. With some established

variable up mostly. This paper proposes a novel approach

techniques, such as the ideas of genetic algorithms and

Number theoretic evolutionary algorithm(NTEA) for

sequential number theoretic optimization to improve the property

global optimization of complex function by combining the

of convergence in large scale, the deflection and stretching of

idea

objective function to guarantee the detection of a different

(SNTO)[3~5] with the concepts from evolutionary

minimizer, it detects optimums of a function through adding

algorithms. This method adds the updating rules with the

genetic operations to the feasible points generated by number

ideas of mutation and other genetic operations to the

theoretic net sequentially. The experiments done indicate that the

feasible points generated by number theoretic net. The

proposed algorithm is robust and efficient.

experiments being done show that NTEA has the

Key words: Optimization, Evolutionary Algorithm, Number

advantages of high precision and high robustness on

Theoretic Net, Mutation, Deflection

multi-model function’s optimization problems to a certain

of

sequential

number

theoretic

optimization

extent. The rest of this paper is organized as follows. In 1 Introduction

Section 2, firstly we introduce the background of our

Detecting global optimums of complex functions is an

research, i.e. details of SNTO and the main idea of

important

fields.

evolutionary algorithms (EA), then the previous work on

Mathematically, complex functions are discontinuous/

extension of SNTO and EA is discussed. We propose a

nondifferentiable/ non-convex full of local optimums. An

novel approach NTEA with established techniques through

extensive literature review [1,2] indicates that most of the

function deflecting and adding evolutionary operations

existing deterministic optimization strategies for complex

and we examine the effectiveness of the novel method by

functions, such as the Newton–family algorithms, are

applying it into classical benchmark functions in Section4.

limited to the optimization of a local area with various

And Section 5 summarizes the paper.

problem

in

diverse

scientific

excess stories using stochastic approaches to globally

2 Backgrounds

Note P

2.1 Number-theoretic methods’ sequential realization Number theory method’s sequence realization SNTO is a Hua L. K. et al. In this method number theoretic net is

M ( t ) = f ( x (t ) ) ≥ f ( y ) , where ∀y ∈ Q ( t ) and

used to generate good points in terms of good lattice sequential contraction the searching field it find the global

x(t), M(t) are the current best value of x*, M*; (t)

(t)

(t)

0

(t)

enough small, c =( b -a )/2, if max c <δ, then

: Suppo ( n; h1 ," , hs )( s < n) is a integer

x(t), M(t) are accept, otherwise go to Step4;

[4]

vector s.t. 1 ≤ hi < n, common

δ >

Step3: Termination condition Judging. Given

optimum theoretically. Definition 1

∈ [a (t ) , b(t ) ] , x ( −1) = {φ} , take

x (t ) ∈ P ( t ) ∪ {x (t −1) } = Q (t ) s.t .

global optimization approach based on number theory by

points scattered in feasible domains, then through

(t )

hi ≠ h j (i ≠ j ) and greatest (n, hi ) = 1, i = 1," , s;

measure

⎧⎪ qki = khi (mod n), k = 1," , n , modify the where ⎨ ⎪⎩ xki = (2qki − 1) / 2n, i = 1," , s

Step4:

Region

contraction.

Define

a new

region

D(t+1)=[a(t+1), b(t+1)] as

⎧ai(t +1) = max( xi(t ) − γ ci( t ) , ai( t ) ), (i = 1, 2," , s ) ⎨ (t +1) (t ) (t ) (t ) ⎩ bi = min( xi + γ ci , bi ), where γ∈(0,1) is pre-given contraction ratio, generally γ=0.5, t =t+1, go to Step1; 2.2 Evolutionary algorithm

≤ n,

Evolution has created wonders like us. There is much to

then the lattice point set of generic vectors ( n; h1 ," , hs )

study of computational systems which use ideas and get

common congruence method to make qki s.t . 1 ≤ ki

defined by Pn

be learned from Nature. Evolutionary computation is the inspirations from natural evolution. The Evolutionary algorithm (EA) [6~9] has been

= {xk = ( xk1 ," , xks ), k = 1," , n}.

widely used for solving optimization problems. In EA

Suppose f(x) is continuous function on bound and

initial population generated by random points in feasible

closed set D , take a NT-net P = { xk , k = 1, 2, , n }

region of problem which is likely to select the fittest

xn∗ ∈ {xk }

the environment survive at higher probabilities in the next

…

and

let

stand

∗

point s.t . M n = f ( xn ) = max f ( xk ) n

1≤ k ≤ n

for

, we expect

by

uniformly

scattered

points

generation is formed through evolutionary operations such as crossover, mutation and saving the best for respective series of these operations. We expect that the number of

The essence of Number theory method is to replace points

generation among those that forms a generation. The next

individuals. Solution searching is pursued by repeating a

∃xn∗ ∈ { xk } s.t . f ( xn∗ ) → M , xn∗ → x∗ (n → ∞) random

individuals so that those proven to have greater fitness for

individual with higher fitness (that is, those closer to

on

optimal solutions) increases as the search make progress.

s-dimensional unit cube. When SNTO is used to solve the

Thereby an optimal solution can be achieved. The above

multi-modal function optimization problem, it requires not

describes the basic concept of EA.

computing f(x)’s derivative at point P but f(x)’s value at P,

2.3 Previous studies

and it’s independent of derivation and the choice of initial

But as the other global optimization method, SNTO is

points.

exhaustive and deterministic, and it fails to jump off local [4]

Now we give the main progress of SNTO as below: Algorithm 1. SNTO :

points, but it doesn’t mine more information from these (0)

(0)

(0)

Step0: Initialization. t=0, D =D, a =a, b =b; (t)

Step1: Generating a NT-net P uniformly distributed on (t)

optima. In beginning of the method, it has a big sample

(t)

[a , b ] by using a number theory method; Step2: Computing the new fitness.

points. So mining these points may generate more effective points globally. Evolutionary algorithms is a kind of self-adaptive heuristic method simulating evolution process effective on

complex functions’ optimization problems, and it’s similar

s.t . M ( t ) = f ( x (t ) ) ≥ f ( y ) , where ∀y ∈ Q ( t )

to SNTO on "Multi-points", thus this gives the idea of

and x (t), M (t) are the current best value of x*, M*;

combining the idea of SNTO with the concepts from

If the objective function f(x) is full of local optimums

evolutionary algorithms. of

and more than one minimizer is needed, we choose

evolutionary calculation technique with number theoretic

another established techniques to guarantee the detection

method have been done in two ways mainly: First, using

of a different minimizer, such as deflection and stretching

number theoretic net to generate initial feasible points for

are introduced. Suppose objective function is f(x), we use

EA [10]; second, parallel strategies [11] for SNTO using

deflection technique as below to generate the new

small initial feasible population and using evolutionary

objective function F(x)[8]:

Some

Previous

studies

of

combination

operator crossover to these point.

k

F ( x) = ∏ ⎡⎣ tanh ( x − xi∗ ) ⎤⎦

3.1 Proposed Technique NTEA Based on the above mentioned studies, We propose a novel number-theoretic evolutionary algorithm (NTEA) by doing evolutionary operations (mainly mutation) on equi-distribution set Pn generated by number theoretic net in SNTO while keeping SNTO’s advantages in our technique, just modifying the step2 of SNTO, the NTEA is given as below in details: Algorithm 2. NTEA Step2: Computing the new fitness.

= P (t ) ∈ [a ( t ) , b(t ) ] , do mutation to

(t )

f ( x)

i =1

3 NTEA and Numerical Experiments

Note P0

−1

where

xi∗ (i=1,2,…,k) are k minimizers founded,

λ∈(0,1). We also introduce stretching technique[8] to generate the new objective functions G(x) and H(x) as new objective functions:

G ( x) = f ( x) + β1 x − xi∗ ⎡⎣1 + sgn( f ( x) − f ( xi∗ )) ⎤⎦ 1 + sgn( f ( x) − f ( xi∗ )) H ( x) = G ( x) + β 2 tanh ⎡⎣δ (G ( x) − G ( xi∗ )) ⎤⎦ where β1, β2, δ > 0. Fig.1. shows deflection and stretching effects on f(x)= cos x at x=π

P0(t ) in probability p1 , for xi ∈ P0(t ) , ⎧ xi ,k + Δ(t , y ), rand = 0 , ⎩ xi ,k − Δ(t , y ), rand = 1

Let xi , k = ⎨

, Δ (t , y ) = y[1 − r

y = bk(t ) − ak(t )

(1− Tt )b

] , T

denotes the algorithm’s largest iteration number, t is the iteration number now, b is coefficient, r=rand [0,1]; (t )

(t )

Then we get P1

, do crossover to P1

as

Fig.1.Deflection and stretching effects on f(x)

below: Suppose x1 , x2 ∈ P1

(t )

[0,

1],

k

then x1, k = 1

P1(t )

;

is

, β is a random number on

random

integer

β x1,k + (1 − β ) x2,k Note

on

[1,

s],

, and now we get

x ( −1) = {φ}

,

x (t ) ∈ P0( t ) ∪ {x (t −1) } ∪ P1(t ) ∪ P2(t ) = Q ( t )

take

In this way, we see that the searching algorithms will not locate x=π. So we can combine this technique with NTEA to find more minimizers needed more efficiently. 3.2 Benchmark functions To check the effective of these algorithms, we performed a series experiments. First we define standard functions [1] introduced mainly by De Jones and often used as quantitative evaluation means for the benchmark test of

EA and other technique. They are defined as below:

point is (0,0), and they are classic benchmark functions to

eg.1. f1(x,y)=21.5+xsin(4πx)+ ysin(20πy), -3.0 ≤

evaluate EA’s global convergence and local exploring

x ≤ 12.1,

4.1 ≤ y ≤ 5.8, its global maximum value is known as [1]

performance.

38.827553 . This is a classical multimodal function whose global

3.3 Comparisons of results among EA, SNTO and

optimum is surrounded with many local points.

For all the testified functions, we take the same generic

NETA

f2 (x,y)=-[20+xsin (9πy) +ycos (25πx)], where (x,y)

vector for number net to generate good points as these:

∈ D= {( x, y ) x + y ≤ 9 } , its minimum value

first cycle we take (987; 1,610), then we take (233; 1,144)

eg.2.

2

2

2

-32.71788780688353, correspond global minim point is [9]

in the latter cycles. To

avoid

non-convergence

for

long-playing

(-6.44002582194051, -6.27797204163553) . Fig.2 is f2

circulation we replace max c(t)<δ by min c(t)<δ; and

on D, where its value out of D is put as -40.

generally take δ=10-15, r=0.6. And we add mutation operator in contraction region. Especially, for f1 we choose mutation ratio 0.05, require the newer optimal not smaller than old one. For f2 we take δ=10-8, r =0.86, mutation ratio=0.85 and we make the function value 3 at points out of defined region and we take a new mutation method as this: Generate a random number α on [0, 1], if α>0.95 or α<0.05, then mutation in defined region; otherwise mutation in correspondent contraction region. For f6, we require the newer optimal not smaller than

Fig.2. f2 on D f2 is a multimodal function difficult to be optimized. Its defini region D is big, xsin (9πy) and ycos (25πx) oscillate in different directions in their ways and it has

old one not strictly. For f7, we take δ=10-30 and require the newer optimal bigger than old one not strictly. Now we use Matlab6.5 to program NTEA and its

deep valley clatters near to 4 points.

performance in contrasted to SNTO, simple evolutionary

eg.3. Easom function f3 (x1, x2) =-cos x1 cos x2

Algorithm SEA. The global search capability, convergence

2

2

exp[-(x1-π) -(x2-π) ], its global minimum is known as -1,

speed and robustness of the algorithms proposed is

correspond global minim point is (-π,π).

exhibited in Fig3~6: Fig.3 is NTEA contrast simulation

eg.4. Bohachevsk1 function

progress of the best, average and worst function value in

f4 (x1, x2) =

x12 + 2 x22 − 0.3cos(3π x1 ) − 0.4 cos(4π x2 ) + 0.7 .

iteration for f1. Fig.4 shows the local convergence of

eg.5. Bohachevsk2 function

contour of Easom. Fig.6 is contrast simulation progress of

f5 (x1, x2) =

x12 + 2 x22 − 0.3cos(3π x1 ) ⋅ cos(4π x2 ) + 0.3 . eg.6. Schaffer function

0.5 +

sin

2

(1.0 + 0.001( x

2 2

40

2 1

+ x )) 2 2

35 Best function avalue 30 Average function avalue

2

. And it is called

Multimodal Sine Envelope Sine Wave Function [2]. eg.7.

the three algorithms–SEA, SNTO and NTEA for f6.

f6 (x1, x2) =

x + x − 0.5 2 1

NTEA for f2. Fig.5 is the progress of optimization in

f7 (x1, x2) =

25 Worst function avalue f1 20

15

10

5

( x12 + x22 )0.25 ⎡⎣1.0 + sin 2 ( x12 + x22 )0.1 ⎤⎦ .

0

0

10

20

30

40

50

60

70

Iteration number

eg.3.~eg.7. are defined in -100≤x1, x2≤100, their global minimum is known as 0, correspond global minim

Fig.3. Comparison of Best, Average, Worst fitness for f1

No.2 [-6.44002582235322, -6.27797201356924], No.3 [-6.4400258222786, -6.2779720141167] No.4 [-6.44002582226788, -6.27797201447922], No.5 [-6.44002582235673, -6.27797201347218], No.6 [-6.44002582213847, -6.27797201162225], No.7 [-6.44002582211448, -6.27797201373397], No.8 [-6.44002582217536, -6.27797201223602], No.9 [-6.44002582208887, -6.27797201222687]

−6.276

−6.2765

−6.277

Y

−6.2775

−6.278

−6.2785

−6.279

−6.2795

From the Table1 and Table2, We could draw a

−6.28

conclusion that the performance of NTEA is much better

−6.2805

−6.442

−6.4415

−6.441

−6.4405

−6.44

−6.4395

−6.439

−6.4385

−6.438

than SNTO and SEA, especially for f1, f2, F6, F7. And for

X

Fig.4. Local convergence of NTEA for f2 in contour map

f4, f5 NTEA does as well as SNTO but superior to SEA increasingly.

10

9

8

1.48

1.4803e−016

Y

016

Cases studies illustrate that NTEA With established

1.4803e−016

7

6

4 Conclusions.

03e−

1.4803e−016

techniques in Section 3. for complex function optimization 1.4

1.4803e−016

−0

16

1.4803e−016

−0.8

1.4803e−016

1.4803e−016

4

. −0

−0

.6 −0.4 −0.2

2

1.4803e−016 1

3e

1.4803e−016

−0.2

4

3

1

can improve the global convergence speed and has the

80

5

2

3

advantages of high precision and robustness to such a certain extent. In this paper we propose a technique in which SNTO

1.4803e−016 4

5

6

7

8

9

10

X

Fig.5. Finding optimal for Easom in contour map

is combined with the concept of evolutionary calculation technique, and verified its effective using standard

0.35

functions. The improved-version NETA has inherited the

0.3

features of both: simple SNTO is strong at raw exploration and EA is robust on multimodal functions. At the end,

0.25

through experiments, NETA gave better performance than

SEA 0.2

either SNTO or EA.

SNTO

f

6

NTEA

In the future works, we will do more researches on

0.15

adjusting the parameters of genetic operations in NTEA

0.1

and their relations to the algorithm’s stability to improve

0.05

algorithm’s robustness and precision much more. And we

0 0

10

20

30

40

50

60

70

80

90

100

Iteration number

Fig.6. Comparison of Best fitness for f6

plan to apply this technique to more complicated and difficult problems. We would like to develop other algorithms that perform well by combining EA’s concepts

Table1 and Table2 are the results of contrasted

with the proposed algorithms.

optimal searched by NTEA, SNTO and SEA and correspondent optimal points to the optimal values in

ACKNOWLEDGEMENT

Table1 searched by NTEA. With deflection technique, we can detect 6 minimizers of f1 with the same value: No.1 [11.6255447026864, 5.72504424431332], No.2 [11.6255447027255, 5.7250442445237], No.3 [11.6255447026949, 5.72504424490065], No.4 [11.6255447027843, 5.72504424471521], No.5 [11.6255447027431, 5.72504424413162], No 6 [11.6255447027118, 5.72504424463833] And 9 minimizers of f2 with the same value are also founded: No.1 [-6.44002582207099, -6.2779720144943],

We thank the anonymous reviewers for their helpful remarks and comments. F. Gao was supported by the Foundation (Grant No.XJJ2004113), Project of educational research, and the UIRT Project (Grant No. A156, A157) granted by Wuhan University of Technology in People's Republic of China.

References [1]

Michalewicz

Z.

Genetic

Algorithms

+Data

Structures=Evolution Programs (3rd. edition).Berlin: [2]

[8]

orbits of nonlinear mappings through particle swarm

Srinivas M., Patnail L. M., Adaptive Probabilities of

optimization, in Proceedings of the 4th GRACM

Crossover and Mutation in Genetic algorithms. IEEE

Congress on Computational Mechanics, Patras,

Trans. on Sys. Man & Cybernetics, Vol., 24, No.4,

Greece, 2002. [9]

Apr.1994, pp. 656～666. [3]

[4]

Parallel

solving a system of nonlinear equations. J. Comp.

Optimization

Math, 1991, 9: pp.9～16.

2001(38): pp.1381～1386（In Chinese）.

LIU B. D., ZHAO R. Q. Statistical Programming Fuzzy

Programming.

Beijing:

Algorithms

for

.Theoretical

Computer

Function Science,

[10] ZHANG Ling, ZHANG Bo, Good Point Set Based

Tsinghua

Genetic Algorithm. Journal of Computer, 2001(24):

University Press, 1998.

[6]

KANG L.S., LIU P., CHEN Y.P., Two Asynchronous

Fang K.T. and Wang Y. A Sequential Algorithm for

and [5]

Parsopoulos K., Vrahatis M., Computing periodic

Springer -Verlag, 1996.

pp.917-922（In Chinese）.

KANG L.S., XIE Y., YOU S.Y. etc. Non-numerical

[11] LIU H.Q., YUAN X.G., FANG K.T. Application of

parallel algorithms: Simulating annealing algorithms.

evolution

Science Press, Beijing, 1997（In Chinese）.

Optimization. Trans. of Tianjin University, 2001(8):

PAN Z.J., KANG L.S., CHEN Y. P. Evolutionary

pp.221-225（In Chinese）.

Sequential

Number

Theoretic

Computation. Beijing: Tsinghua University Press, 1998（In Chinese）. [7]

Hua L. K. and Wang Y. Applications of Number theory to Numerical analysis. Springer -Verlag & Science Press, Berlin & Beijing, 1981.

Table1 Optimum searched by NTEA, SNTO and SEA and known Value Test Function

NTEA

SNTO

SEA

Known Value

f1

38.8502944794474

38.8502944794474

38.827553

38.827553

f2

-32.7178878068835

-32.4245444521219

-32.4396133897864

-32.71788780688353

f3

-1

-1

-0.99988661032628

-1

f4

0

0

0.00204985947697

0

f5

0

0

0.00596033477647

0

f6

0

/

0

0.07035405596053

0

f7

1.69641423986802×10

0.00971590987751 -16

5.91645474391892×10

-14

Table2 Correspond Optimal point for the optimum in Table1 searched by NTEA Test Function

Correspondent optimal points in Table1 searched by NTEA

f1

(11.6255447027118, 5.72504424463833)

f2

(-6.44002582208887, -6.27797201222687)

f3

(3.14159264305311, 3.14159265381822)

f4 f5 f6 f7

Unknown (-6.44002582194051, -6.27797201463553) (-π,π)

-9

-9

(-3.00171076217092×10 ,-6.14027470452886×10 ) -10

-11

(-1.5805059471004×10 ,1.65931843809982×10 (-2.87741108834877×10

-32

(0,0)

-9

(0,0)

)

(0,0)

-34

(0,0)

(-8.27493200622298×10 ,-2.02504829295479×10 ) -8

Correspondent Known point

,-4.85869247033871×10 )