Proceedings of the ASME 2009 Dynamic Systems and Control Conference DSCC2009 October 12-14, 2009, Hollywood, California, USA

DSCC2009-2694

A METHOD FOR DISTRIBUTED OPTIMIZATION FOR TASK ALLOCATION Sheng Zhao, Baisravan HomChaudhuri and Manish Kumar Dept. of Mechanical Engineering University of Cincinnati Cincinnati, Oh, 45221 [email protected]

ABSTRACT Allocation of a large number of resources to tasks in a complex environment is often a very challenging problem. This is primarily due to the fact that a large number of resources to be allocated results into an optimization problem that involves a large number of decision variables. Most of the optimization algorithms suffer from this issue of non-scalability. Further, the uncertainties and dynamic nature of environment make the optimization problem quite challenging. One of the techniques to overcome the issue of scalability that have been considered recently is to carry out the optimization in a distributed or decentralized manner. Such techniques make use of local information to carry out global optimization. However, such techniques tend to get stuck in local minima. Further, the connectivity graph that governs the sharing of information plays a role in the performance of algorithms in terms of time taken to obtain the solution, and quality of the solution with respect to the global solution. In this paper, we propose a distributed greedy algorithm inspired by market based concepts to optimize a cost function. This paper studies the effectiveness of the proposed distributed algorithm in obtaining global solutions and the phase transition phenomenon with regard to the connectivity metrics of the graph that underlies the network of information exchange. A case study involving resource allocation in wildland firefighting is provided to demonstrate our algorithm.

different tasks, so that the overall objective is achieved in an optimal manner, is a very challenging problem. This is primarily due to the complexity that arises from a large number of decision variables. Furthermore, dynamic and uncertain nature of environment makes optimization problem more challenging. Additionally, in a complex and dynamic environment, the volume of information needed to reach the global solution could be huge, and obtaining an optimal solution could be computationally prohibitive. This paper proposes a distributed greedy optimization method to reduce computational load and hence shorten response time. A lot of effort has been made to address the task allocation problem in the Distributed Artificial Intelligence community [7, 8]. There are many auction-based algorithms for the multi-robot task allocation problem such as M+ [4] and MURDOCH [5]. The comparison of computational complexity between those methods can be found in paper [6]. Market-based algorithm is one of the promising ways to allocate resources [9]. Market-based algorithm is inspired from the field of economy where each resource is a self-interested individual whose only purpose is to maximize its benefit. Through a careful design of reward and cost for each task, the actions by the self-interested agents can help obtain an optimal solution that is close to the global one. Auction is a commonly used method in market-based algorithms to help the agents negotiate with each other and allocate themselves to different tasks. One of the most popular market-based architectures is Hoplites [10]. In this architecture, there are two types of coordination: passive coordination and active coordination. The agents passively coordinate with each other until the best profit they can get is not acceptable due to the existence of constraints. Then they start actively generating team

INTRODUCTION The allocation of resources is encountered in various domains such as manufacturing, operation management, firefighting, disaster management, and multi-robot cooperation [1-3]. They often represent complex decision making problems. To effectively use and allocate a large number of resources to

1

Copyright © 2009 by ASME

plans to escape local minima. This feature makes it very suitable for constrained task allocation scenarios. Threshold-based algorithm is another popular distributed method for optimization which is inspired from some of the swarming behaviors seen in ants [11, 12]. For instance, ants are very simple insects with limited intelligence and ability to process information. They use only limited local information without direct communication with other ants. However, they exhibit many complex group behaviors such as clustering, foraging and labor division. Response threshold model has been proposed in literature [12] to explain the task allocation and labor division in ant colonies. In this model, each task has own stimulus level and each agent has a threshold. If the stimulus of a task is higher than agent threshold, the agent will perform this task. When an agent is performing a task, the stimulus of this task will decrease. Every agent will give up its task after certain time to avoid being stuck in one task. The objective here is to minimize the stimulus intensity.

FIGURE 1. FIRE MODEL

Given the number of resource assigned to a fire site (ni) and the area already burnt (Areai), we can find an optimal fire line which can minimize the area burnt by fire. The method to find the optimal fire-line that we have developed is based upon a Genetic Algorithm which is outside the scope of this paper. As the resources build the fireline, the area under fire continues to grow before completely extinguishing. The increment of the burnt area is a function of initial burnt area and the number of resources assigned to it, and can be denoted as:

In this paper, we propose a distributed greedy algorithm to allocate resources. One of the major features of distributed algorithms that lend them their abilities to find solutions is sharing of information and cooperation. Graphs have been used to represent the information flow in a network. The vertices represent the locations or tasks and edges represent the available information paths. In this paper we try to study how the some of the properties of the underlying graph impacts the time taken by the algorithm to converge to a solution and the ability of the algorithm to reach global solutions. The wildland firefighting problem is chosen as a case study to illustrate our algorithm. Simulation results are given to demonstrate the effectiveness of distributed algorithms in solving complex resource allocation problems. Furthermore, simulation results are presented to study the effect of graph properties (which govern the information flow) on the performance of the algorithm.

Areai  f ( Areai , ni )

(1)

The function f can be any function that fits the model here. In this paper it is assumed to be as:

f ( Areai , ni ) 

Areai ni  1

For multiple fire locations, the objective is to minimize the total increment of the burnt area (Fig. 2).

PROBLEM FORMULATION Wildland firefighting presents a complex task allocation problem where the incident commander needs to allocate a large number of resources to various fire sites in a very dynamic and uncertain environment. Usually, during high fire-season, multiple numbers of distinct sites catch fire. Incident commander needs to allocate resources to all sites so that a certain objective function, such as area burnt, is minimized. Usually, these resources are used to build a fire-line which is a corridor that surrounds the wildfire where fuel is eliminated so that fire cannot cross. So, given an initial area of fire and the total number of available resources, the objective of the task allocation problem is to use the resources to minimize the whole fire burnt area (Fig. 1).

FIGURE. 2. MULTIPLE FIRE SITES

Min  Area i  f ( Area1 , n1 )  f ( Area 2 , n2 )  ...  f ( Area n , nn ) (2)  subject to  ni  N total 

With the objective function given by Equation (2), we consider the task allocation problem under two assumptions: #Assumption 1: All the agents or resources have the global knowledge of the fire sites. This means that every agent knows the reward and cost provided in each fire site.

2

Copyright © 2009 by ASME

For Assumption 2: In this assumption, each agent can only know the information about the local fire sites. A graph is used to represent the topology of information flow here (Fig. 4). Graph theory has been traditionally used to study and model networked systems. An undirected graph G is a pair of sets V , E , where

#Assumption 2: The agents only have local knowledge of the fire sites. Each agent can only be aware of the situation in its immediate neighborhood defined by edges of the graph representing the information flow. DISTRIBUTED GREEDY STRATEGY

V  1, 2,  n is the set of n nodes and E  V  V is the set

For Assumption 1: The distributed greedy approach, proposed in this paper, is a utility based algorithm in which we first assign rewards and costs to every fire sites. In this algorithm, we assume each resource as an agent which makes the decision where to go. Once the rewards and costs are set, the rule for each agent is to choose the site which provides the biggest profit. The profit an agent can get jumping from site j to site i is defined as:

profiti  reward i  cost j

of edges that connect two nodes i.e. an unordered pair of distinct vertices. Each node or vertex represents a task location or fire site. The edges represent the interactions between the task locations and are formed due to flow of information between the task locations. Each node has a set of neighbors that is defined by N i   j  V : i, j   E . The neighborhood relations in a graph are captured by adjacency matrix A which is defined by:

1 i, j   E Aij   0 otherwise

(3)

The reward provided to the next agent that comes to site i, which has ni number of currently allocated resources and Areai as initial burnt area, is:

reward i  f ( Areai , ni )  f ( Areai , ni  1)

A number of quantities defined in graph theory such as degree matrix, and Laplacian are used to analyze various properties of graph (such as connectivity). The degree matrix of

(4)

th

the graph G is a diagonal matrix D with the i diagonal element n

Similarly, the cost for one agent that leaves a site j is: equal to

cost j  f ( Area j , n j  1)  f ( Area j , n j )

(6)

A j 1

(5)

ij

which is the row sum of adjacency matrix A. The

i th diagonal element of the degree matrix represents the degree of th the i node. The Laplacian L of graph G is given by L  D  A . The Laplacian of a graph is a symmetric and

At the beginning of the algorithm, all the resources stay at one site. The agents, then, sequentially jump from their current site to the site which provides them the largest profit (Fig. 3). If an agent jumps, it gets reward from the site it jumps to. But it needs to pay the cost to the site where it jumps from. Every agent chooses the site providing the biggest profit. This process continues until the agents cannot find any site which will provide them positive profit.

positive semi-definite matrix and provides information about several properties of the graph including edge and vertex connectivity. For example, the eigenvalues of the Laplacian matrix provides information about the number of (disconnected) components of the network [13]. We apply the same algorithm we used for the assumption 1 for this scenario except that a resource assigned to any site can have information about the neighboring sites only and can jump to only these sites. It can be expected that the convergence time and the quality of solution will depend upon the connectivity of the network graph. In this paper, we try to answer the questions such as what kind of connectivity properties of the graph can lead the algorithm to reach the global minima, and how fast the algorithm can converge to the solution.

FIGURE. 3. DISTRIBUTED GREEDY ALGORITHM

3

Copyright © 2009 by ASME

0 1  0 A 0 0   0

1 0

0 1

0 1

0 0

1

0

0

1

1 0

0 1

0 1

1 0

0

1

0

0

0 0  1  0 0  0 

The algorithm takes 7 iterations to converge to the allocation solution given by the vector: [15, 19, 9, 20, 17, 20]. The value of the cost function is 9.9524 which is a sub-optimal one. The final configuration is shown in Fig. 5.

FIGURE. 4. GRAPH REPRESENTING THE INFORMATION FLOW AMONG THE FIRE SITES

SIMULATION RESULTS Example 1 In this example, we consider the total number of fire sites to be six, and the total number of resources that need to be allocated to be 100. Numbers randomly selected from a uniform distribution in the range 5 to 50 were chosen as the initial fire burnt area value, and found to be [20, 35, 10, 40, 30, 50] where element of the matrix represents initial burnt area for respective fire site.

FIGURE. 5. THE FINAL RESOURCE ALLOCATION FOR EXAMPLE 1 USING ASSUMPTION 2

Example 2 As shown in the previous example, we can expect that, due to the communication constraints, the algorithm may get stuck in local minima, and we may not obtain the optimal solution under this assumption. Obviously, the performance of our algorithm largely depends on the connectivity of the graph. In the following part of the section, we study the effect of graph connectivity on the performance of our algorithm.

In the first assumption, these fire sites are fully connected so that information flows from each site to every other site. Our market based algorithm converges to the optimal solution in one iteration. In other words, after each agent makes its jumping decision, the resulting solution is the globally optimal one. The allocation result obtained by the algorithm is [14, 18, 9, 20, 17, 22]. To ensure that the solution obtained was global, we carried out the optimization using the Genetic Algorithm. The optimal cost function value is 9.9208.

Let us first define the graph connectivity. In this paper, we use the algebraic connectivity, i.e., the second smallest eigenvalue of Graph Laplacian L as a measure of connectivity of the graph. If the graph is connected, the second smallest eigenvalue of L is always a positive quantity greater than zero. The magnitude of this connectivity value is related to how well connected the graph is, and this has been widely used to study several properties of graphs [14].

In the second assumption, the fire sites are not fully connected, and each fire site has only limited number of neighboring sites with which it can interact. The network topology for these fire sites considered in this example is shown in Fig. 4. A resulting adjacency matrix, A, that represents the connection in the graph is given by:

In this example, we increase the complexity of problem space. We consider that there are 100 fire sites and 1000 homogeneous resource to be allocated. We randomly generate 100 values ranging between 5 to 50, then assign these values to fire sites as initial fire burnt area. Fig. 6 shows the value we assigned to each fire sites.

4

Copyright © 2009 by ASME

For assumption 1, the algorithm reached the global minima still in single iteration. The final cost function was 567.8047 which matches the global solution obtained from the Genetic Algorithm. The final allocation solution is shown in Fig. 7. We carried out extensive simulation studies with different initial fire burnt area conditions. In each of the simulations, the algorithm converged to the global minimum solution in just one iteration. These simulations point to the fact that with the global knowledge, we can easily reach the global solution in a short time.

We randomly generated 1200 graphs (i.e., graphs with randomly chosen adjacency matrix) with different connectivity. Fig. 8 shows how the error decreases with respect to the increase in the algebraic connectivity. If we draw a curve using curve fitting toolbox in matlab, we can clearly observe that the error drops sharply to 0 as the connectivity goes up (Fig. 8). When the connectivity goes beyond 3, the error will almost be 0. Another critical metric of the algorithm’s performance that we studied was the time it took to arrive at the solution. Since the time taken for the algorithm is proportional to the number of iteration it takes to converge, we studied the number of iterations. A plot of the number of iterations vs. connectivity is shown in Fig. 9. As we can see, the iteration time decreases as we increase the connectivity. As we can see from these results, the connectivity plays an important role on the performance of the algorithm. The higher the connectivity is, the more convenient and efficient is sharing the information in a network. Another important property that can be seen from the Figures 8 and 9 is the emergence of a phenomenon. For example, it can be seen that there is drastic change in error values when the connectivity is around 0.25. Similarly, in Figure 9, the number of iterations required changes drastically at around the connectivity of 5. In fact such emergence of phenomenon has long known to be existing as phase transition phenomenon in complex networks [15].

FIGURE. 6. INITIAL FIRE BURNT AREA

FIGURE. 7. NUMBER OF RESOURCES ALLOCATED TO FIRE SITE LOCATIONS

For assumption 2, we randomly generated the adjacency matrix (thereby randomly choosing the information flow topology), and then looked into how connectivity affected the performance of the algorithm. Global solution (global optimal cost function) was obtained using Genetic Algorithm, which was also the same as the solution obtained using assumption 1, i.e., with the full connectivity. We define the error as the difference in cost functions between the solution obtained from our algorithm under assumption 2 and the global solution. It may be noted that the solution obtained from our algorithm under assumption 2 would always have a cost function value greater than or equal to that obtained from the global solution.

FIGURE. 8. ERROR VS. CONNECTIVITY

5

Copyright © 2009 by ASME

[4] Botelho, C.S., and Alami, R., 1999. “M+: A Scheme for Multi-robot Cooperation through Negotiated Task Allocation and Achievement”. In Proc. Of the IEEE Intl. Conf. on Robotics and Automation (ICRA), May, pp. 1234-1239. [5] Gerkey, B.P., and Mataric, M.J., 2002. “Sold!: Auction Methods for Multirobot Coordination”. IEEE Transactions on Robotics and Automation, Oct, pp. 758-768 [6] Gerkey, B.P., and Mataric, M.j., 2003. “Multi-Robot Task Allocation: Analyzing the Complexity and Optimality of Key Architectures”. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Sept, pp. 3862-3868.

FIGURE. 9. ITERATION TIME VS. CONNECTIVITY

[7] Gerkey, B.P., and Mataric, M.J., 2004. “A Formal Analysis and Taxonomy of Task Allocation in Multi-Robot Systems”. The International Journal of Robotics Research, Vol. 23, No. 9, pp. 939.

CONCLUSION In this paper, we developed a greedy distributed algorithm inspired from market based optimization techniques for task allocation and demonstrated its effectiveness in wildland firefighting example. The proposed distributed resource allocation strategy was evaluated for two assumptions: 1) when information available to each agent was global, 2) when information available to each agent was local. Using the metrics derived from graph theoretic representations of information networks, we evaluated the performance of our algorithm in terms of the ability to reach the global solution, and the time taken to reach the solution. Simulation results demonstrate the effectiveness of our proposed algorithm in reaching the global solution, and the effect of algebraic connectivity on the performance of the algorithm. We also noted the emergence of phase transition phenomenon characteristic of complex networks. Future work will focus on developing effective methods to reach a global allocation solution under communication constraints. Market-based algorithm with explicit communication could be a promising method to approach this goal.

[8] Peng, C., and Kumar, V., 2008. “An Almost CommunicationLess Approach to Task Allocation for Multiple UAVs”. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), May, pp. 1384-1389. [9] Dias, M.B., Zlot, R., Kalra, N., and Stentz, A., 2006. “Market-based Multirobot Coordination: A Survey and Analysis”. In Proceedings of the IEEE, Vol. 94, No. 7, July, pp. 1257-1270. [10] Kalra, N., Ferguson, D., and Stentz, A., 2005. “Hoplites: A Market-based Framework for Planned Tight Coordination in Multirobot Teams”. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), April, pp. 1170-1177. [11] Kalra N., and Martinoli A., 2006. “A Comparative Study of Market-based and Threshold-Based Task Allocation”. In Proc. of the Eight Int. Symp. on Distributed Autonomous Robotic Systems, July, pp. 91-102.

REFERENCES [1] Jones, E.G., Dias, M.B., and Stentz, A., 2007. “Learningenhanced Market-based Task Allocation for Oversubscribed Domains”. In Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS), Nov, pp. 2308-2313.

[12] Campos, M., Bonabeau, E., Theraulaz, G., and Deneubourg, J.L., 2000. “Dynamic Scheduling and Division of Labor in Social Insects”. Adaptive Behavior, Vol. 8, No. 2, pp. 83-95. [13] Olfati-Saber, R., 2006. “Flocking for Multi-Agent Dynamic Systems Algorithms and Theory”. IEEE Transactions on Automatic Control, March, pp. 401-420.

[2] Schrage, D., Farnham, C., and Gonsalves, P.G., 2006. “A Market-based Optimization Approach to Sensor and Resource Management”. In Proceedings of the International Society for Optical Engineering (SPIE), Jun, pp. 62290I.

[14] Olfati-Saber, R., 2005. “Ultrafast consensus in small-world networks”. In Proceddings of American Control Conference, Jun, pp. 2371- 2378 vol. 4. [15] Frank, J., and Martel, U.C., 1995. “Phase Transitions in the Properties of Random Graphs”. Principles and Practice of Constraint Programming, Aug, pp. 62-69.

[3] Michael, N., Zavlanos, M.M., Kumar, V., and Pappas, G.J., 2008. “Distributed Multi-robot Task Assignment and Formation Control”. In Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS), May, pp. 128-133.

6

Copyright © 2009 by ASME