Jorge M Pacheco1, 2 Program for Evolutionary Dynamics, 1 Harvard University, 1 Brattle Square, 6th, 02138 Cambridge, MA USA 2 CFTC and Departamento de Física da Universidade de Lisboa, Av. Prof. Gama Pinto, 2, 1649-003 Lisboa Codex, Portugal

I start by investigating how norms may have evolved in the early days of human history, where human societies were mostly organized in small groups or tribes. I define a charicatural model of a world in black and white, in which norms evolve through a novel multilevel selection process. A simple, ubiquitous norm – stern judging – emerges, which combines prompt forgiveness with implacable punishment. As observed at other levels of organization, “simplicity is the key to success” in the evolution of norms. Subsequently, I explore the fact that real populations, in general, are strongly heterogeneous, such that some individuals have many more and more often contacts than others. This fact contrasts with the traditional homogeneous setting used in mathematical studies of Darwinian evolution. Heterogeneity in the population is incorporated by studying games on graphs. I study the evolution of cooperation, modeled in terms of the most popular dilemmas of cooperation, in which mutual cooperation is undermined by greed and fear. For all dilemmas, cooperators manage to outweight defectors whenever the populations exhibit broad, scale-free like structure. 1. introduction Natural selection is conventionally assumed to favor the strong and selfish who maximize their own resources at the expense of others. But many biological systems, and especially human societies, show persistent patterns of altruistic, cooperative interactions. Therefore, one may naturally wonder: How can natural selection promote unselfish behavior? Here I shall explore two (out of the various) mechanisms invoked to resolve such a conundrum, related to two distinct forms of interactions between individuals. The first one deals with cooperation based on reputation, in which groups of individuals evolve in a world in which each group is dominated by a given social norm, and from time to time one group tries to impose its own norm to another group of individuals. Studies of so-called indirect reciprocity have a long history1, often related to group selection, until very recently a tabu-concept in evolutionary biology2. A recent and excellent review on the subject has been published by Martin Nowak and Karl Sigmund in Ref.1. The present model, however, and unlike most studies carried out to date, introduces multilevel selection assuming the existence of different evolutionary games operating at different levels of

132

darwin, evolution and cooperation

selection. Although individuals evolve in groups, groups evolve autonomously by means of a frequency dependent (group) selection determined by a distinct game – details in section2. Indirect reciprocity is believed to constitute the mechanism which distinguishes us, humans, from all other animal species1. Yet, cooperation is routinely observed in interactions between individuals from other species. In fact, simpler levels of cooperation are supposed to be on the basis of some of the most important transitions in evolutionary history3. Simple mechanisms of cooperation are usually modeled, mathematically, in terms of so-called social dilemmas of cooperation, such as the celebrated prisoner’s dilemma, the snowdrift game or the stag-hunt game4. Up to very recently, however, such simple mechanisms of cooperation assumed infinite populations in which every single individual would interact with all her peers, a feature which, while mathematically solvable in the mean-field approximation, is by no means realistic. Indeed, one may envisage that in small groups every individual interacts with her peers. Yet, evolutionary game theory in small, finite populations, exhibits features which may deviate dramatically from the infinite, well-mixed populations so popular throughout the evolutionary studies of the last two decades5, 6. Changes of this paradigm were introduced in the early nineties, where studies of social dilemmas were carried out in populations arranged on regular lattices, such that each individual would interact solely with her nearest neighbors7, 8. Such “spatially extended” models showed how “population viscosity” might favor cooperation in the prisoner’s dilemma, but admittedly such a modeling framework suffers from severe shortcomings, when one tries to accommodate living populations under such a crystallized, military-parade type of socio-biological layout. By the turn of the twentieth century, such shortcomings became obvious with the advent and rapid development of network theory9, 11. Clearly, population structures are best represented by heterogeneous graphs, which perfectly capture the essential features of what one observes in populations ranging from bacteria to insects and humans. Indeed, compelling evidence has been accumulated that a plethora of natural, social and technological real-world networks of contacts (NoCs) between individuals are strongly heterogeneous, different individuals engaging in different patterns of interactions, often exhibiting scale-free behavior9-11, as illustrated in F1. Here I examine how cooperation evolves whenever individuals interact following heterogeneous NoCs, engaging in single rounds of a social dilemma characterized by given intensities of greed and fear4. No analytic solution exists for any of the problems I address in this paper, except perhaps in mean-field approximations which, as will become clear, fail at times to capture even the essentials. Therefore, most of my discussion will find support in results from agent-based simulations. Part of the results reported here have been published already12-18.

jorge m. pacheco

133

F1 Heterogeneous versus Homogeneous NoCs. a) Scale-free graph, built following the Barabási-Albert model10.

The thick black edges illustrate the direct ties which link the most connected individuals. b) Homogeneous regular graph, in which every individual is equivalent to any other, exhibiting a degree distribution characterized by a single peak. The limit z = N-1 leads to a complete graph. Histograms: Degree distributions, computed for each type of graph and N N N=104. In both cases the average connectivity z is 4. The colouring of vertices illustrates one possible realization at the start of a simulation (see section 3), in which 50% of the nodes are randomly populated with cooperators (blue circles) and the remaining with defectors (red circles).

2. reputation-based cooperation: from the pleistocene to the internet Indirect reciprocity presumably provides the mechanism which distinguishes us, humans, from all other living species on Earth1. Adopting the terminology resulting from the seminal work of Hamilton, Trivers, and Wilson19-21, an act is altruistic if it confers a benefit b to another individual in spite of accruing a cost c to the altruist (where it is assumed, as usual, that b>c). Unlike direct reciprocity, in which repeated altruistic acts between the same two individuals lead to a mutual net benefit22, under indirect reciprocity alone, any two players are supposed to interact at most once with each other, one in the role of a potential donor, while the other as a potential receiver of help. Each player can experience many rounds, but never in the same role with the same partner twice, direct retaliation being unfeasible. By helping another individual, however, a given player may increase (or not) her reputation, which may change the pre-disposition of others to help her in future interactions. Her new reputation, however, depends on the social norm used by her peers to assess her action as a donor. Such a reputation-based model of cooperation has been thoroughly explored, as reviewed recently1. It was concluded that cooperation outweights defection whenever assessment of actions is based on norms which distinguish justified defection from unjustified defection1, a feature which demands considerable cognitive capacity, even when individuals are capable of making binary assessments only, in a world in black and white1, as assumed in most recent studies1, 14, 23-26. Furthermore, stable cooperation relies27 on the availability of reliable reputation information.

134

darwin, evolution and cooperation

F2 Norm Complexity. The higher the complexity of a norm, the more “inner” layers it acquires. The outer layer stipulates the

donor’s new reputation based on the 3 different reputation/action combinations aligned radially layer by layer: Inwards, the first layer defines the action of the donor. The second defines the reputation of the recipient; the third the reputation of the donor. Out of the 28 possible norms, the highly symmetric norm shown as the outer layer emerges as the most successful norm (see section 2). Indeed, stern-judging renders the inner layer (donor reputation) irrelevant in determining the new reputation of donor.

Binary reputations lead to well-defined hierarchical norms. The simplest are the so-called first order norms, in which all that matters is the action taken by the donor. In second order norms the reputation of the recipient also contributes to decide the new reputation of the donor. And so on, in increasing layers of complexity (and associated requirements of cognitive capacities from individuals) as illustrated in F2. Previous studies have investigated norms up to third order. As illustrated in 2, I use the same norm-length here. Justified defection contrasts with technology-based interactions, such as e-trade, which also rely on reputation-based mechanisms of cooperation28-30. Indeed, anonymous one-shot interactions between individuals loosely connected and geographically dispersed usually dominate e-trade, raising issues of trust-building and moral hazard30. Reputation in e-trade is introduced via a feedback mechanism which announces rating of sellers. Despite the success and high levels of cooperation observed in e-trade, it has been found27 that publicizing a detailed account of the seller’s feedback history does not improve cooperation, as compared to publicizing only the seller’s most recent rating. In other words, practice shows that simple reputation-based mechanisms are capable of promoting high levels of cooperation. In view of the previous discussion, we may well ask: How can evolution explain such a conundrum?

jorge m. pacheco

135

model Let us consider a world in black and white consisting of a set of tribes, such that each tribe lives under the influence of a single norm, common to all individuals (see F3). Such a setting might be associated with the Pleistocene during which, presumably, humans spread through most of the world. As I will argue below, the conclusions obtained in such a model of primitive societies accounts for the recent findings on e-trade alluded to before.

F3 Multilevel selection model for the evolution of norms. Each palette represents an island in which inhabitants (coloured

dots) employ different strategies (different colours) to play the indirect reciprocity game. Each island is influenced by a single social norm (common background colour), which may be different in different islands. All individuals in each island undergo pair-wise rounds of the game, whereas all islands also engage in pair-wise conflicts, as described in the main text. As a result of the conflicts between islands, norms evolve, whereas evolution inside each island selects the distribution of strategies which best adapt to the ruling social norm in each island.

In each island, each individual engages in a single round of the give-and-receive game1 with every other island inhabitant, assuming with equal probability the role of donor or recipient. The donor decides if YES or NO she provides help to the recipient, following her individual strategy. If YES, then her payoff decreases by 1, while the recipient’s payoff increases by b>1. If NO, the payoffs remain unchanged14. Since reputations are either GOOD or BAD, there are 24=16 possible strategies. On the other hand, the number of possible norms depends on their n associated order. For a norm of order n there are 22 possible norms, each associated with a binary string of length 2n. We consider third order norms (8 bit-strings, F2): In assessing a donor’s new reputation, the observer has to make a contextual judgment involving the donor’s action, as well as his and the recipient’s reputations. The action of a donor will be witnessed by a third-party individual who,

136

darwin, evolution and cooperation

based on the island’s social norm, will ascribe (subject to some small error probability µa = 0.01) a new reputation to the donor, which is assumed to spread efficiently without errors to the rest of the individuals in that island23-26. After all interactions take place, one generation has passed, simultaneously for all islands. Individual strategies in each island replicate to the next generation in the following way: For every individual A in the population we select an individual B proportional to fitness (including A). The strategy of B replaces that of A, apart from a small mutation probability µs = 0.01. As usual under natural selection, the most successful individuals in each island have a higher reproductive success. Since different islands are “under the influence” of different norms, the overall fitness of each island will vary from island to island, as well as the plethora of successful strategies which thrive in each island (F3). Evolutionary dynamics proceeds in alternating waves of war and peace. Peace time has just been described. War-time corresponds to islands engaging in pair-wise conflicts with a small probability pWAR=0.1. After each conflict, the norm of the defeated island will change towards the norm of the victor island. Assume a complete graph for the Network of Contacts (NoC) of each island, i.e., every island is directly connected to every other island. (The same is assumed for the individuals in each island, consistent with the idea of small tribes – see F3). With probability pWAR, each island engages in a conflict with any other island. If island A goes to war, then we choose at random its adversary from the remaining islands. Let us call it B. Each island has accumulated a mean payoff GDPA and GDPB respectively, as a result of the “give and receive” game played by individuals in each island. For each island there are two possible strategies, HAWK and DOVE, similar to the Hawk-and-Dove game described in31. We define payoff values V for “victory”, T for the investment of each player when both decide to play DOVE, W for the cost of fighting for the winner and for the cost of fighting for the loser, such that L>W>0, V>W>0, L+W>V>2T>0 which, as discussed in31 provides the most interesting scenario (for details, cf. ref.14). Assuming that islands are rational players, island A will play HAWK with a probability q directly related with the Nash equilibrium of the game’s payoff matrix, and DOVE with probability (1-q). Similarly for island B. After conflict, the norms adopted by islands A and B will possibly change from what they were before. Let Q(A) be the payoff obtained by A and Q(B) that obtained by B as a result of the game. Then: ·

If A played HAWK and Q(A) > Q(B), then the 8-bit norm NB=NB(7)…NB(0), will change according to: with probability

with probability ·

if

and is mutated by µN«1 if

. The parameter η∈[0,1]

jorge m. pacheco

· · ·

137

is the “will for change”, such that (1-η) acts as an inertia for changing. The population (and its individual strategies) also changes in this case, eliminating at random a fraction ƒ=η’Q(A)/ηη Q(A)/ Q(A)+(1-η’)Q(B)) (η’∈[0,1]), of individuals of the defeated island, replacing Q(A)/ηη them (again at random) by individuals (and their strategies) from the victor island. Same as before, swapping A and B. If A played HAWK and Q(A)≤Q(B) or A played DOVE, then norm entries NB(i) are mutated with probability µN«1 and the population strategies are mutated by µS. Same as before, swapping A and B.

This provides an updating rule for the norms and for the population, and after this step we are back to peace, evolution being dominated by individual interactions within each island. During “peace” time, norms are kept constant. The war-and-peace evolutionary cycle is repeated until it reaches a stationary state. results Extensive computer simulations were performed, involving 64 tribes with 128 inhabitants in each tribe (for details, cf. ref.14). Each simulation runs for 9000 generations, starting from randomly assigned strategies and norms, in order to let the system reach a stationary situation, typically characterized by all islands having maximized their average payoff, for a given benefit b > c=1. The subsequent 1000 generations are then used to collect information on the strategies used in each island and the norms ruling the islands in the stationary regime. We ran 200 evolutions for each value of b, subsequently performing a statistical analysis of the bits which encode each norm. We compute the frequency of occurrence of bits 1 and 0 in each of the 8 bit locations. To this end we use all norms from all islands from all runs (1000x200), and we consider that a bit fixates if its frequency of occurrence exceeds 98% (no changes occur if we use 95% instead). Otherwise, no fixation occurs, which we denote by “X” “X”, instead of “1” or “0” “0”. Subsequently, we compute the frequency of occurrence ϕ1,ϕ0 and ϕX of the bits “1”, “1” “0” and “X” “X”, respectively. If ϕ1 >ϕ0 + ϕX the final bit is 1; if ϕ0 > ϕ1 + ϕX the final bit is 0; otherwise we assume it is indeterminate, and denote it by “•”. The results for different values of b are given in Table 1, showing that a unique, ubiquitous social norm emerges from these extensive numerical simulations. T1 Norm dependence on benefit. For each value of b, we show the results obtained for the resulting Norm, obtained as

described in main text. Irrespective of whether a given bit is fixed or not, all Norms are consistent with norm 10011001 for all values of b.

138

darwin, evolution and cooperation

For all values of b we may write it as (1001)2 (that is, the 8-bit string is obtained via the two-fold concatenation of the 4-bit string 1001 – see F2). This reflects the fact that the norm which emerges is of second-order, which means that all that matters is the action of the donor and the reputation of the receiver. In other words, even when individuals are equipped with high cognitive capacities, they opt for a simple norm as a key for evolutionary success. Indeed, simplicity is the key for evolutionary success. discussion The success of this simple norm relies on never being morally dubious: To each type of encounter, there is one GOOD move and a BAD one. Moreover, it is always possible for anyone to be promoted to the best standard possible in a single move. Conversely, one bad move will be readily punished with the reduction of the player’s score. This prompt forgiving and implacable punishment leads me to call this norm stern-judging. Within the space of third-order norms explored until now1, only the image-score norm proposed by Nowak and Sigmund27 constitutes a second-order norm. Unlike image-scoring, however, stern-judging constitutes one of the two most successful leading eight norms discovered by Ohtsuki & Iwasa24,26 within the space of third-order norms. Moreover, under stern-judging, cooperative strategies flourish. The present result correlates perfectly with the recent findings in e-trade, where simple reputation-based mechanisms ensure high levels of cooperation. Indeed, this norm involves a straightforward and unambiguous reputation assessment, which I argue is often on the basis of e-trade assessment, where potential customers are devoid of constraining environments, their decision of buying or not buying being free from further ado. Finally, stern-judging also allows one to arrive at a precise definition of GOOD and BAD actions. If we impose a clear definition of GOOD and BAD behaviours and also that GOOD and BAD individuals should be treated differently, we end up with only two possibilities, namely 1001 or 0110. At any level of description, only the 1001 norm is able to promote cooperation, thereby becoming viable fitness-wise, justifying a posteriori the association of GOOD with 1 and BAD with 0, respectively. 3. simple cooperation: evolution in heterogeneous populations Here I turn to cooperation between individuals who are endowed with a fixed strategy, are not subject to any norm, and play with their neighbours a simple dilemma of cooperation in which reputation plays no role. Considering simpler games allows me to assess more transparently the role played by population structure and heterogeneity in the evolution of cooperation. I shall conveniently map a given population onto a graph (see F1), in which individuals (agents) occupy the vertices and their patterns of interactions – the Network of Contacts (NoCs) – are defined by the edges linking the vertices12-18. At the most elementary level, social dilemmas can be formalized in terms of symmetric two-person games based on two choices – to cooperate (C) or to defect (D). These two choices lead to four possible outcomes: CC, CD, DC and DD. With each outcome, a particular payoff is associated:

jorge m. pacheco

139

R (reward) and P (punishment) are the payoffs for mutual cooperation (CC) and defection (DD), respectively, whereas S (sucker) and T (temptation) are the payoffs associated with cooperation by one player and defection by the other, respectively. Several social dilemmas arise naturally4, depending on the relative ordering of these four payoffs, obeying the following constraints: i) R > P: players prefer mutual cooperation (CC) over mutual defection (DD). ii) R > S: players prefer mutual cooperation over unilateral cooperation (CD). iii) 2R > T + S: players prefer mutual cooperation over an equal probability of unilateral cooperation and defection. iv) T > R: players prefer unilateral defection (DC) to mutual cooperation or P > S: players prefer mutual defection to unilateral cooperation (CD). Dilemmas will exhibit different degrees of tension between individual and collective interests, based on the above relations. Given that R > P, tension becomes apparent when the preferred choices of each player lead to individual actions resulting in mutual defection, in spite of the fact that mutual cooperation is more beneficial. The extent to which such individual actions occur may be adjusted introducing different intensities of greed (the temptation to cheat, whenever T > R), of fear (of being cheated, whenever P > S) or both, leading to three well-known social-dilemma games: The Snowdrift Game (SG), for which T > R > S > P, where tension is due to greed but not fear, the game of Stag Hunt (SH), for which R > T > P > S, where tension results from fear but not greed, and the Prisoner’s Dilemma game (PD) game, in which both fear and greed are present, that is, T > R > P > S. Although these are the most popular social dilemmas, other games arise in other areas of this four-dimensional parameter space, as illustrated in F4. In the following we simplify the problem by normalizing the advantage of mutual cooperation over mutual defection, in all games, to the same base value, making R=1 and P=0. With this choice for R and P, we are left with two parameters, T and S. Depending on their values, these parameters may add (or not) different intensities of greed, fear or both to each game. We study the behaviour of all dilemmas in the ranges 0≤T≤2 and -1≤S≤1, which will be shown to be sufficient to characterize the games under study, fear being present whenever S<0 (S>0 may be interpreted as courage) while greed is present whenever T>1 (T<1 may be interpreted as generosity). Note further that, in the SG, only the lower triangle illustrated in F4 strictly satisfies generosity the requirement that 2R > T + S.

140

darwin, evolution and cooperation

F4 Dilemmas of cooperation. Fixing R=1 and P=0 reduces the four-dimensional space of symmetric two-player games to a twodimensional space. We explore the square 0≤T≤2 and -1≤S≤1. Different areas of the (S,T)-plane are associated with different “dilemmas”. We consider the Prisoner’s Dilemma (PD), Stag-Hunt (SH) and Snowdrift Game (SG). In infinite, well-mixed populations, the only stable equilibrium for the PD is 100% defectors. For the SG, 100% cooperators or defectors are unstable equilibria, and a stable equilibrium exists for a fraction of cooperators given by S/(S+T-1). The same relation holds for the SH, but now the equilibrium is unstable, whereas 100% cooperators or defectors become stable equilibria.

networks and simulation details games on graphs In the language of graph theory, well-mixed populations of size N are represented by complete graphs, which correspond to a regular, homogeneous graph with average connectivity z=N-1 (F1-b), since all vertices share the same number of connections. Indeed, all homogeneous graphs exhibit the same shape for the degree distribution d(k), defined for a graph with N vertices as d(k) = Nk / N, where Nk gives the number of vertices with k edges (F1), reflecting the topological equivalence of all vertices. Real-world NoCs, on the other and, are clearly heterogeneous, corresponding to populations in which different individuals exhibit distinct patterns of connectivity, portraying the coexistence of local connections (spatial structure) with non-local connections (or shortcuts) and often exhibiting a power-law dependence of their degree distributions10,11. The Barabási-Albert10 model provides the best-known model leading to distributions d(k) ~ k-γ, with γ=33 (F1-a). The construction of a scale-free graph using the Barabási-Albert model involves two processes: 1. Growth:: Starting with a small number ((m0) of vertices, at every time step we add a new vertex with m ≤ m0 edges that link the new vertex to m different vertices already present in the system; 2. Preferential attachment: When choosing the vertices to which the new vertex connects, we assume that the probability pi that a new vertex will be connected to vertex i depends on the degree ki of vertex i: pi= ki / ∑kki . Preferential attachment corresponds to the well-known “rich get richer” effect in economics33, also known as the “Matthew effect” in sociology34. After t time steps this algorithm produces a graph with N = t + m0 vertices and mt edges. Because vertices

jorge m. pacheco

141

appear at different moments in graph-generation time, so-called age-correlations10,11 arise. In order to single out the role of heterogeneity in evolution, we may remove any correlations (including age-correlations) by subsequently exchanging, randomly and repeatedly, the ends of pairs of edges of the original graph35, a procedure which washes out correlations without changing the scale-free degree-distribution. evolution For R=1, P=0, 0≤T≤2 and -1≤S≤1, evolution was carried out implementing the finite population analogue of replicator dynamics13, to which simulation results converge in the limit of homogeneous, well-mixed populations. This corresponds to define the following transition probabilities: In each generation, all pairs of individuals x and y, directly connected, engage in a single round of the game, their accumulated payoff being stored as Px and Py, respectively. Whenever a site x is updated, a neighbour y is drawn at random among all kx neighbours; then, only if Py > Px the chosen neighbour takes over site x with probability given by (P Py-P Px)/[k )/[k>D>], where k>=max(K Kx ,Ky) and D>=max(T,R,S,P)-min(T,R,S,P). simulations Simulations were carried out on graphs with N =104 vertices and average connectivity z=4 (except in connection with F5, where where z=N-1). Equilibrium frequencies of cooperators and defectors were obtained, for each value of T and S, by averaging over 1000 generations after a transient time of 10000 generations. Furthermore, final data results from averaging over 100 realizations of the same type of NoCs specified by the appropriate parameters (N N and zz). All simulations start with an equal percentage of strategies (cooperators and defectors) randomly distributed among the elements of the population. Moreover, even when graphs are generated stochastically, the evolution of cooperation is studied in full grown graphs, that is, the number of vertices and edges is conserved throughout evolution. results and discussion F5 shows the results of our simulations for all social dilemmas as a contour plot. The underlying NoCs correspond to complete, fully connected graphs, which provide the finite population analogue to the infinite, well-mixed limit well-known from the standard analytical treatment. In particular, the results confirm the i) dramatic fate of cooperators under the simultaneous action of greed and fear (PD); ii) a similar fate for cooperators in the absence of greed (SH) whenever generosity does not overshoot fear and iii) the coexistence of cooperators and defectors in the absence of fear, such that cooperators increasingly dominate the higher the intensity of courage (SG). Replacing the well-mixed ansatz for the population by a heterogeneous population exhibiting a scale-free degree distribution such that all connections between individuals are purely random (see methods), leads to the results shown in F6-left). Notice that, in spite of the abundance of scale free behaviour identified in real NoCs, the detailed cartographic representation of the patterns of connections between individuals remains to a large extent unknown32.

142

darwin, evolution and cooperation

F5 Evolution of cooperation in complete graphs. Results for the fraction of cooperators in the population are plotted as a

contour, drawn as a function of the intensity of greed and fear which characterizes a given dilemma. In the absence of greed and fear (upper left square) cooperators trivially dominate. Fear without greed leads to the SH game (lower left square), greed without fear leads to the SG (lower triangle in upper right square), and when both fear and greed are present we obtain the PD game (lower right square). Results were obtained in complete NoCs, the finite population analog of infinite well-mixed populations. These results provide the reference scenario with which the role of population structure will be assessed.

In this sense, random scale-free populations provide a general bias-free scenario. The results in F6-left) evidence the determinant role played by population structure on the evolution of cooperation for all dilemmas. Using F5 as reference we observe that, overall, scale-free NoCs efficiently neutralize the detrimental role of greed in the evolution of cooperation, whereas fear remains a strong deterrent of cooperation. Indeed, under greed alone (SG) cooperators dominate for all values of greed. Under fear alone (SH) cooperation becomes now more likely for small intensities of fear. For the PD the domain of coexistence between cooperators and defectors is clearly broadened.

Moreover, the small slope of the borderline between cooperators and defectors provides further evidence that fear constitutes the major threat to cooperation. The net results (shown in F6-left) hide in fact a detailed interplay of two mechanisms, related to the small-world and heterogeneous nature of the underlying scale-free NoCs: The occurrence of many long-range connections (so-called shortcuts) in scale-free graphs precludes the formation of compact clusters of cooperators, thereby facilitating invasion by defectors. However, the increase in heterogeneity of the NoCs opens a new route for cooperation to emerge16,17, since now different individuals interact different number of times per generation, which enables cooperators to outperform defectors. In other words, while on one hand the increased difficulty in aggregating clusters of cooperators would partially hamper cooperation, heterogeneity, on the other hand, counteracts this effect, with a net increase of cooperation.

jorge m. pacheco

143

F6 Evolution of cooperation in scale-free NoCs. We use the same notation and scale as F5. Left: Random scale-free NoCs.

The interplay between small-world effects and heterogeneity effects, discussed in the main text, leads to a net overall increase of cooperation for all dilemmas. Right: Barabsi-Albert scale-free NoCs. Whenever age-correlations are retained, highlyconnected individuals become naturally inter-connected and cooperators dominate defectors for all intensities of greed, enlarging the range of intensities for which they successfully survive defectors under the action of fear.

The scale-free NoCs of Barabási and Albert10 help us demonstrate how one may go beyond the scale-free properties of given NoCs with the purpose of increasing cooperation. Indeed, if we do not randomize the pattern of connectivity between individuals, such that the NoCs exhibit the correlations arising naturally in the Barabási and Albert model, a different result emerges for the evolution of cooperation, as (shown in F6-right). As is well-known, this model exhibits so-called age-correlations, in which the older vertices not only become the ones acquiring highest connectivity, but also they become naturally interconnected with each other. In other words, the formation of compact clusters of cooperators which was inhibited by the occurrence of many shortcuts in random scale free NoCs, will be partly regained in such NoCs, mostly for the few individuals which exhibit high connectivity. Of course, such a clustering of cooperators will only occur to the extent that cooperators are able to occupy such highly connected sites, which indeed happens17. The results (in F6-right) show that now greed poses no threat to cooperation, defectors being wiped out from populations under greed alone (SG). Under fear alone (SH), cooperators now wipe out defectors where before (F6-left) they managed to coexist. Under the joint action of greed and fear (PD), cooperators also get a strong foot-hold up to larger intensities of fear. The present results show that inclusion of realistic population structure in evolutionary game theory restores cooperation as a competitive evolutionary trait, being more competitive the more heterogeneous the pattern of interactions of a given population. Furthermore, re-formulating three well-known dilemmas in terms of the relative intensities of greed and fear allows a unified analysis of all dilemmas, showing the relative importance of greed and fear as deterrents of cooperation. Finally, by understanding the mechanisms which ensure the sustainability of cooperation, it is possible to conceive specific interaction patterns with the purpose of promoting cooperation. At any rate, fear is a much stronger deterrent of cooperation than greed, a feature which is well-supported by empirical evidence on many biological species, including humans.

144

darwin, evolution and cooperation

4. conclusions In this work I have shown how heterogeneous networks promote cooperation to an evolutionary viable trait. This is a remarkable result, taking into account that, for several decades, cooperation has remained an evolutionary conundrum. On the other hand, whenever one investigates the evolution of social norms by means of an indirect reciprocity game of cooperation, one finds that individuals organized in tribes are better off by adopting simple norms of cooperation. Even when individuals are given the possibility of evolving more sophisticated norms, evolution leads to the emergence of a simple norm which is never morally dubious – stern-judging. Under stern-judging, helping a good individual or refusing help to a bad individual leads to a good reputation, whereas refusing help to a good individual or helping a bad one leads to a bad reputation. Similarly to tit-for-tat (or win-stay, loose-shift), the simplest ubiquitous strategies in direct reciprocity1, the straightforward lack of ambiguity of stern-judging supports the idea that simplicity is often the key for evolutionary success. acknowledgements Most of the results reported here are the product of ongoing collaborations with Francisco C. Santos, Fabio C. Chalub, João F. Rodrigues and Tom Lenaerts. To all my deepest gratitude. I would also like to acknowledge useful discussions with Martin A. Nowak, Arne Traulsen and Hisashi Ohtsuki and Christoph Hauert. Finally, financial support from FCT is gratefully acknowledged. bibliography 1 NOWAK, M. A.& SIGMUND, K. Evolution of Indirect Reciprocity. Nature 437, 1291-1298 (2005). 2

SOBER, E. & WILSON, D.S. (1998) Unto Others: The Evolution and Psychology of Unselfish Behavior, (Harvard University Press, Cambridge, MA).

3

SMITH, J. M., SZATHMÁRY SZATHMÁRY, E. The Major Transitions in Evolution, (Freeman, Oxford)(1995).

4

MACY , M. W., FLACHE, A,. Learning dynamics in social dilemmas. Proc. Natl. Acad. Sci. (USA) 99, 7229 (2002).

5

NOWAK, M. A., SASAKI, A., TAYLOR, C. & FUDENBERG, D., Emergence of cooperation and evolutionary stability in finite populations. Nature 428, 646–-650 (2004).

6

PACHECO, J. M., TRAULSEN, A. & NOWAK, M. A., Evolutionary dynamics in finite populations under pairwise comparison (submitted). submitted submitted).

7

NOWAK, M. A. , MAY, R. M., Evolutionary games and spatial chaos, Nature 359, 826-829 (1992).

8

HAUERT, C., DOEBELI, M., Spatial structure often inhibits the evolution of cooperation in the snowdrift game, Nature 428, 643-646 (2004).

jorge m. pacheco

9

145

AMARAL, L.A.N. , SCALA, A., BARTHELEMY, M. & STANLEY, H.E., Classes of Small-World Networks, Proc. Natl. Acad. Sci. (USA) 97, 11149-11152 (2000).

10 ALBERT, R. , BARABÁSI, A. L., Statistical mechanics of complex networks, (2002) Rev. Mod. Phys. 74, 47. 11 DOROGOTSEV, S. N., MENDES, J. F. F F. Evolution of Networks: From Biological Nets to the Internet and WWW (Oxford University Press, Oxford) (2003). 12 PACHECO, J. M., SANTOS, F. C., Network dependence of the dilemmas of cooperation in Science of Complex Networks: from Biology to the Internet and WWW, Ed. Mendes, J. F. F. (AIP Conference Proceedings) 776, 90-100 (2005). 13 SANTOS, F. C., PACHECO, J. M., Scale-Free networks provide a unifying framework for the Emergence of Cooperation, Physical Review Letters 95, 098104 (2005). 14 CHALUB, FABIO A. C. C., SANTOS, F. C., PACHECO, J. M., The Evolution of Norms, Journal of Theoretical Biology (2006, in press). 15 SANTOS, F. C., RODRIGUES, J. F., PACHECO, J. M., Graph topology plays a determinant role in the evolution of cooperation, Proc. Royal Soc. London B273, 51-55 (2006). 16 SANTOS, F. C., RODRIGUES, J. F., PACHECO, J. M., Epidemics spreading and cooperation dynamics on homogeneous small-world networks, Physical Review E 72, 056128 (2005). 17 SANTOS, F. C., PACHECO, J. M., A new route to the evolution of cooperation, Journal of Evolutionary Biology (2006, in press). 18 SANTOS, F. C., LENAERTS, T., PACHECO, J. M., Evolutionary Dynamics of Social Dilemmas in Structured Heterogeneous Populations, Proc. Natl. Acad. Sci. (USA) (2006, in press). 19 HAMILTON, W. D. Narrow Roads of Gene Land Vol.1 (Freeman, New York, 1996). 20 TRIVERS, R. Social Evolution (Benjamin Cummings, Menlo Park,1985). 21 WILSON, E. O. Sociobiology (Harvard Univ. Press, Cambridge, Massachusetts,1975). 22 TRIVERS, R. The evolution of reciprocal altruism. Q.Rev.Biol. 46, 35-57 (1971). 23 BRANDT, H. & SIGMUND, K., BRANDT, H. & SIGMUND, K. The logic of reprobation: Assessment and action rules for indirect reciprocity. J. Theor. Biol. 231, 475–486 (2004). 24 OHTSUKI, H. & IWASA, Y., How should we define goodness?—reputation dynamics in indirect reciprocity, J. Theor. Biol. 231, 107-120 (2004). 25 BRANDT, H. & SIGMUND, K., Indirect reciprocity, Image scoring and Moral Hazard, Proc. Nat. Acad. Sci. (USA) 102 2666-2670 (2005).

146

darwin, evolution and cooperation

26 OHTSUKI, H. & IWASA, Y., How should we define goodness?—reputation dynamics in ndirect reciprocity (Erratum), J. Theor. Biol. 232, 451 (2005). 27 NOWAK, M. A. & SIGMUND, K. Evolution of indirect reciprocity by image scoring. Nature 393, 573-577 (1998). 28 DELLAROCAS, C. Sanctioning reputation mechanisms in online trading environments with moral hazard. MIT Sloan School of Management working paper no. 4297-03 khttp: //ssrn.com/abstract1⁄4 393043l (2003). 29 BOLTON, G. E., KATOK, E. & OCKENFELS, A. How effective are electronic reputation mechanisms? An experimental investigation.Manage.Sci.50,1587-1602(2004). 30 KESER, C. Trust and reputation building in e-commerce. IBM-Watson Research Center, CIRANO working paper no.2002s-75khttp://www.cirano.qc.ca/en/publication_ detail.php?id1⁄4 2002s-75l (2002). 31 CROWLEY, P. H, Hawks Doves and Mixed-symmetry Games, J. Theor. Biol., 204, 543-563 (2000). 32 GUIMERÀ, R. AMARAL, L. A. N., Functional cartography of complex metabolic networks, Nature 433, 895 (2005). 33 SIMON, H., On a class of skew distribution functions, Biometrika 42, 425 (1955). 34 MERTON, R. K., The Matthew effect in science , Science 15 9, 56 (1968). 35 MASLOV, SERGEI & SNEPPEN, KIM, Specificity and stability in topology of protein networks, Science 296, 910-912 (2002).