The Evolution of Behavior in Biased Populations Daniel H. Wood∗ July, 2015
Abstract I consider how cognitive biases affect the evolution of behavior. In my model, a population of non-Bayesians repeatedly are matched together to play 2 × 2 coordination games. Members of the population have systematically biased beliefs about the distribution of strategies in the population, to which they noisily best respond. Their cognitive biases lead players to make more simultaneous errors than Bayesian players would, changing the evolutionary dynamics. For a large class of biases, the long-run outcome is unchanged from the Bayesian outcome, but behavior can evolve much more quickly as a result of correlated errors. JEL Classification: C72, C73, D03, D83. Keywords: Evolution, stochastic stability, heuristics and biases, false consensus effect, fast convergence, waiting times.
∗
Department of Economics, Clemson University, 228 Sirrine Hall, Clemson, SC 29634. Email:
[email protected]. Phone: 1-864-656-4740. Fax: 1-864-656-4192. An earlier version of this paper circulated as “Cognitive Biases as Accelerators of Behavioral Evolution”. The suggestions of two anonymous referees significantly improved this paper. I also thank Neil Calkin, Glenn Ellison, Eric Mohlin, Tom Mroz, Heinrich Nax, Bill Sandholm, Ryoji Sawa, Rajiv Sethi, Patrick Warren, and Dai Zusai for helpful advice, and audiences at the 2014 Stony Brook Game Theory Festival, 2013 SABE/IAREP/ICABEEP conference, and Temple University for their feedback.
1
1
Introduction
Over the past thirty years much attention has been paid by economists to biases in probability judgement and how they affect decision-making, but less attention has been paid to how these biases affect how aggregate behavior evolves in strategic settings. I analyze the evolution of behavior in populations in which errors stem from two sources: first, there is a small probability that players do not make optimal strategy choices given their beliefs about how other players are acting, and second, players are not Bayesians in forming these beliefs. Each player places too much weight upon one or more observed strategies – his focal observations – and consequently places too little weight upon the other strategy choices in the population. I show that the presence of these biases generally does not change the long-run evolutionary outcome, but that convergence to that outcome can occur much faster than in a population of Bayesians. In my model n players repeatedly play a coordination game with pairwise random matching, noisily best-responding to their beliefs about the other players’ strategies, as in Kandori, Mailath and Rob (1993). Players generate new beliefs each period based on observing the exact history of play that period, but treat r strategy observations differently than the remaining n − r observations. They assign weight (1 − η)/n to every strategy observation but also assign an additional weight η/r to their focal strategy observations. For instance, some players, influenced by the false consensus effect, could believe the population to be strategically more similar to themselves than it actually is, so that they assign additional weight to their own strategy choices. Alternatively or in addition, some players could overweight the same (random) focal observation each period because that observation “sticks out” for some reason. This overweighting corresponds to the availability heuristic, a heuristic in which people estimate probabilities of events by how easy it is to recall 2
instances of the event. Some players could have idiosyncratic random focal observations, corresponding to the representativeness heuristic, which causes people to treat small samples from a population as more representative of the population than they actually are. I describe these biases further in Section 2 and use them as examples throughout, but my results apply to all non-Bayesian belief formation processes that do not explicitly favor one strategy over the other. Loosely speaking, the additional weights on focal observations cause strategy choices that are not the best response to the population’s aggregate behavior – “errors” – to be positively correlated either within a period or over time. Because players are playing a coordination game, when a player’s focal observations are errors, she is more likely to make one herself. If players share focal observations, then “clusters” of errors occur in some periods. If players have different focal observations, more errors in the current period make it more likely that players, by overweighting errors, make errors in the next period as well. Exactly how these correlations change the evolutionary dynamics is the focus of this paper. My paper makes two contributions. While there is considerable evidence that many people are biased in some ways, to my knowledge no research exists which addresses how biases would affect stochastic evolutionary models. My first contribution is to present a tractable model that encompasses a variety of probability judgement biases and makes sharp predictions about the long-run behavior of biased populations. I show that as long as biases are not too severe, they do not explicitly favor a particular strategy, and players occasionally make optimization errors, then the risk-dominant equilibrium is uniquely stochastically stable.1 1
In 2 × 2 games, the risk-dominant equilbrium is to play the strategy which provides a higher payoff against an opponent who is equally likely to play either strategy (Harsanyi and Selten 1988). This well-known result was first shown by Young (1993) and Kandori, Mailath and Rob (1993), using techniques developed in Freidlin and Wentzell (1998) and initially applied to evolutionary dynamics by Foster and Young (1990). While most of my results are for 2 × 2 games, I show that in m-strategy games, 12 -dominant equilibrium – a generalization of risk-dominance due to Morris et al. (1995) – will typically be selected.
3
This result applies to a large category of unrelated biases and mixtures of biases, so if one doubts that people are perfect Bayesians, but is agnostic about exactly how people are biased, this result is still informative.2 My second contribution is to show that optimization errors – the typical assumption about the source of noise in stochastic stability models – and errors in beliefs are qualitatively different. Populations prone to both kinds of errors converge much more quickly to the risk-dominant equilibrium as a result of their erroneous beliefs. Ellison (2000) shows that for small enough probability of optimization error , convergence to the stochastically stable state can take O(−cn ) periods, where c is a constant derived from the payoffs of the game, which can be extremely slow.3 The speed of evolution is generally limited by how often players make sufficiently many simultaneous errors to move the system between basins of attraction of states corresponding to each pure-strategy equilibrium of the underlying game. The positive correlation between errors produced by biases in beliefs leads to lower order convergence times as η increases for fixed population sizes n. The asymptotic behavior as n grows large depends on the exact form the biases take; for 2 × 2 games I show that, given moderate levels of bias, convergence times are bounded as n → ∞ under the false consensus effect and the availability heuristic, while under the representativeness heuristic convergence times are bounded as n becomes large provided that optimization errors are frequent enough. I am unaware of any models that address how biased beliefs affect the evolution of behavior in strategic situations. A few papers address the related question of the longrun behavior of population with cognitive limitations. Sethi (2000) examines the dynamic 2
Existing experimental evidence, while limited, suggests that simple error models do not fit observed behavior very well (Maes and Nax 2014). This suggests behaviorally-inspired models may be a fruitful avenue to explore. 3 If f (x) is of order g(x), written O(g(x)), as x → 0, then ∃C, x ¯ such that | f (x) |< C | g(x) | for all x ∈ (0, x ¯).
4
stability of S(1) equilibria (Osborne and Rubinstein 1998), an equilibrium concept based on a model of procedural rationality in which players sample the effect of playing each action once. Khan and Peeters (forthcoming) investigates long-run outcomes in adaptive play with two populations where the populations may have different levels of cognition, and finds that the stochastically stable states may change as a result of higher sophistication. Although not focused on psychological biases, Blume (2003) probably comes closest to my equilibrium selection question. Blume shows that noisy strategy revision processes that are skew-symmetric – roughly, noise processes where strategy labels do not matter – lead to risk-dominant equilibria being stochastically stable. This was later generalized by Norman to dynamics with simultaneous strategy revisions (Norman 2009b). These results suggest my selection results for 2 × 2 games (Proposition 1). However, a fundamental assumption of this literature is that as noise vanishes the probability of a sub-optimal strategy switch falls to 0 exponentially, which does not occur around the mixed strategy equilibrium in my framework due to errors in belief (see Maruta (2002) for a detailed analysis generalizing skew-symmetry). Several varieties of stochastic evolutionary model address the problem of long wait times. Evolution on networks or where populations have some form of local interaction often is faster (for instance Ellison (1993) or Montanari and Saberi (2010)). For some applications these structual assumptions are hard to justify, though, and this paper instead focuses on evolution in populations that interact globally. Given that focus, the papers closest to this one are Binmore and Samuelson (1997) and Kreindler and Young (2013), which show that adding additional noise leads to waiting times of lower order than the standard model. Binmore and Samuelson combines optimization errors with a learning model that is an additional source of noise. The noise from the learning
5
process makes convergence to the stationary distribution fast as → 0. My paper analyzes an opposite source of noise, but mechanically the biases in my model are akin to learning in Binmore and Samuelson. Kreindler and Young instead consider a logit response dynamic where is small but non-vanishing and show that convergence is fast if one strategy is sufficiently less risky (in my model’s terms, p∗ is low enough) and noise is high enough. The mechanism generating fast transitions is essentially serial correlation similar to the representativeness heuristic in this paper. Recently Ellison, Fudenberg and Imhof have extended Kreindler and Young’s results to m-action games and provided results on the relationship between sampling and fast convergence similar to how very biased populations with serial correlation behave in my model as n → ∞ (Ellison, Fudenberg and Imhof 2014). My work complements these papers on both substantive and technical levels. Substantively, I address a much different question than any of these papers: how do well-documented behavioral biases change the evolution of behavior. Technically, both Binmore and Samuelson (1997) and Kreindler and Young (2013) are birth-death models that allow simple closed-form solutions for the steady-state distribution but limit the analysis from being extended to non-2 × 2 games, which would be possible in my framework. Ellison, Fudenberg and Imhof (2014) provides tools for m-strategy games, but my characterization of long-run dynamics is much more detailed than those authors’ comparable results and focuses on the small- limit in addition to the large-n limit. In addition, many papers consider evolution when players best-respond to a sample of observations of play. In Young (1993)’s adaptive play dynamics, players respond to a sample of the recent strategies that other players recently played; , in Young’s model, sampling serves to avoid deterministic cycles. Oyama, Sandholm and Tercieux (forthcoming) consider
6
deterministic best response dynamics where a continuum of players randomly samples a random number of observations and show that when samples of no more than size k are likely enough, the population converges quickly to an iterated k1 -dominant equilibrium. Focalobservation belief formation generalizes sampling in these papers. Another related rationale for faster transitions is the presence of switching cost that players incur for changing strategies. Norman (2009a) shows that these costs produce intermediate limit states similar to those that occur with the false consensus effect in my model. In both cases, these limit states cause faster transitions due to the faster nature of step-by-step evolution initially described by Ellison (2000). Finally, several recent papers combine biases and evolutionary game theory in particular environments. Sawa (2012) derives the stochastically stable state in a two-stage Nash demand game with outside option when players obey prospect theory. Mohlin (2012) and Heller (forthcoming) consider the evolution of cognitive types with different levels of reasoning (a la Stahl and Wilson (1994) or Costa-Gomez et al. (2001)).
2
Biases in Probability Judgement
Before proceeding with my model, I describe three well-known probability-judgement biases which the model applies to. Of course there are other plausible – and many implausible – ways that people may fall short of the Bayesian ideal, such as confirmation bias (ignoring data not consistent with prior beliefs) or the false uniqueness effect (essentially the opposite of the false consensus effect). Most of my results would apply to these other biases as well.
Representativeness Heuristic: Kahneman and Tversky first introduced the representativeness heuristic, sometimes referred to as the “law of small numbers” (Tversky and 7
Kahneman 1971, 1974). It is a probability estimation heuristic where biased individuals take small samples from a population as more representative of that population than the samples truly are. Rabin (2002) is a contemporary model of how the bias affects individual behavior while Kahneman (2011) is a popular exposition. The representativeness heuristic has been most studied in settings where news arrives over time, such as investors evaluating stock performance. In that case, biased individuals under-react (relative to Bayesians) to short-run trends, because they under-estimate the probability of sequences with high variance and consequently expect mean-reversion. However, they over-react to longer trends, because they take these sequences as more informative than they are about the underlying direction of the market. This pattern is consistent with the real-world patterns of short-run momentum and long-run reversals in stock returns (Rabin and Vayanos 2010; Barberis et al. 1998). Similarly, Barber et al. (2009) show that individual investors’ trading is positively correlated across time and across investor, which they explain as (partially) produced by overextrapolation of past returns due to the representativeness heuristic. My model predicts similar positive correlation stemming from the representativeness heuristic.
False consensus effect: The false consensus effect refers to an egocentric bias in which people overestimate how much other people’s judgements are similar to their own (Ross 1977; Marks and Miller 1987; see also Dawes 1989; Vanberg 2008a on problems with earlier studies of this effect). For instance, students take their own performance on a test of ‘social sensitivity’ as more informative about overall pass rates on the test than another student’s performance (Alicke and Largo 1995). Several studies have found that the the false consensus effect plays an important role in simple games where social preferences influence behavior. Blanco, Engelmann, Koch and 8
Normann find that in sequential prisoner’s dilemmas, first-movers who cooperate are more likely to cooperate reciprocally as second-movers. Much of this behavior is caused by a subject’s second-mover decisions influencing their beliefs about what other subjects would choose in the same situation, in addition to the direct influence of subject altruism on both decisions (Blanco et al. 2014). Other researchers have found similar belief-driven correlations in subjects’ choices in trust games (Vanberg 2008b; Ellingsen et al. 2010) and in a sequential voluntary contributions game (G¨achter et al. 2012). Yet other studies directly test the false consensus effect by asking student subjects to predict a peer group’s decisions. They find predictions consistent with subjects believing themselves to be more typical than they are (Proto and Sgroi 2013; Engelmann and Strobel 2012; but see also Engelmann and Strobel (2000)).
Availability Heuristic: People following the availability heuristic judge how probable an event is based on how easy it is to recall examples from memory (Tversky and Kahneman 1974). In a classic study, Schwarz et al. asked subjects to list 12 examples of assertive behaviors on their part. These subjects rated themselves as less assertive than subjects who were only asked to list 6 examples. Constructing a list of 12 examples is difficult for most people, and subjects treated the difficulty of the task as providing information that they were not assertive. Indeed, subjects who are asked to list 12 unassertive examples judge themselves to be more assertive than subjects who are asked to list 12 assertive examples (Schwarz et al. 1991). Research on the availability heuristic in economic decision-making is more limited. Kuran and Sunstein argue that the availability heuristic is important to understanding risk regulation and that “availability entrepreneurs” try to manipulate public discourse to take advantage of the heuristic (Kuran and Sunstein 1999). 9
3
Model
Players in a even-numbered population N = {1, 2, . . . , n} recurrently play the simultaneousmove coordination game in Figure 1 with uniform random matching. (A, A) and (B, B) are Nash equilibria along with a mixed-strategy equilibrium in which A is played with probability
p∗ =
d−b . a−c+d−b
I assume throughout that a − c > d − b so that p∗ < 1/2, and (A, A) is risk-dominant. At the end of the period, every player i ∈ N receives a strategy revision opportunity. In revising her strategy, player i first forms a belief pˆi,t about the population frequency of period-t A choices, which will generally differ from the true frequency pt . I refer to players as “Bayesian” if they use the information they have available optimally in constructing pˆi,t , and “non-Bayesian” if they do not use the information optimally.4 Then she chooses her strategy for the next period, si,t+1 . Players are myopic in that they respond to pˆi,t and do not take into account that the populations’ behavior might change in the future. Let BR(p) be a best-response to the population distribution p and and W R(p) be a non-best-response. 4
It may strike readers as absurd to call simple myopic players in a stochastic evolutionary model “Bayesian”. The distinction I am drawing is whether pˆi,t is as accurate as possible, and not whether the beliefs are then used optimally.
A
B
A
a, a
b, c
B
c, b
d, d
Figure 1: Coordination Game. a > c, d > b, a − c > d − b.
10
Then si,t+1 = BR(p) with probability 1 − and W R(p) with complementary probability. If i chooses si,t+1 = 6 BR(pt ) she makes a revision error (or simply “error”). A revision error can stem from two sources. It can follow from an error in beliefs: pˆi,t inaccurate enough that BR(ˆ pi,t ) 6= BR(pt ). It can also occur due to an optimization error: si,t+1 = W R(ˆ pi,t ). For games with two strategies, if i simultaneously makes both optimization and belief errors, the two errors cancel each other out and she does not make a choice error. I focus on populations where each member forms their beliefs using what I call focalobservation belief-formation processes. These are ones in which the player observes the entire vector st but in forming pˆi,t places too much weight on r players’ observed strategy choices and too little weight on the true frequency of players (including herself) playing A. Player i’s focal observation set records which players player i overweights in period t. Let Fr , the set of all possible focal player combinations, be the set of all r-combinations of N , and let ∆|Fr | be the |Fr |-dimensional simplex. A particular bias assigns a probability to i overweighting the behavior of each combination of players. Definition 1. Player i’s bias at time t, Bi,t ∈ ∆|Fr | gives the probability that i’s focal observation set Fi,t is a given element of Fr . In forming pˆi,t , i overweights all j ∈ Fi,t equally. Let NA,t = |i ∈ N : si,t = A| be the set of players using strategy A at t. Abusing notation, fi,t is equal to the fraction of players in Fi,t who are also in NA,t . It is a random variable with support on p ∈ (0, 1r , . . . , 1) Definition 2. A player i who forms his beliefs about the fraction of players playing A with a focal-observation belief-formation process with bias Bi,t has beliefs
pˆi,t = ηfi,t + (1 − η)pt
11
where η ∈ [0, 1] parameterizes the severity of the players’s bias and fi,t is a random variable equal the fraction of players in Fi,t who are in NA,t . For example, under the false consensus effect (FCE), each player’s own strategy is focal for herself, so r = 1 and every Bi,t is such that Pr(Fi,t = {i}) = 1. Under the availability heuristic (AH) and the representativeness heuristic (RH), focal observation sets are uniform random draws from Fr . Let Zr be a uniform distribution over Fr . For representativeness each player overweights a different small sample (of size r ≥ 1), so Bi,t = Zr , i.e., Fi,t ∼ Zr . For availability, imagine r strategy choices are broadcast to the entire population each period and players with the availability heuristic then overweight these announced choices. There is one focal-observation set Ft ∼ Zr shared by all players, so ∀i, Fi,t = Ft . These biases are well-known examples from the psychology literature, but for most of my results I do not restrict all Bi,t to be one of these biases or even that members of the population share similar biases. All that I require of my populations’ bias functions is that (i) each member has a bias that satisfies a “no directional bias” property under which strategy labels do not affect how focal-observation sets are formed and (ii) the population’s set of biases on aggregate lead to sufficiently random focal observation sets. Assumption 1 (No directional bias). All players’ bias functions Bi,t are such that the expected frequency of strategies in their focal observation sets is equal to the true frequency of strategies: E[fi,t | pt ] = pt . For instance under the false consensus effect, if si,t = A then pˆi,t = η + (1 − η)pt , while for si,t = B, pˆi,t = (1 − η)pt . Therefore
E[fi,t |pt ] = Pr(si,t = A)(η + (1 − η)pt ) + Pr(si,t = B)(1 − η)pt = pt
12
satisfying no directional bias. Assumption 2 (Aggregate randomness). Players’ biases are such that (i) if at ≥ r, then either (a) there are at + 1 players for which Pr(Fi,t ⊂ Na,t ) > 0 or (b) there are at players for which Pr(Fi,t ⊂ Na,t ) = 1, and (ii) if at ≤ n − r, then either (a) there are at − 1 players for which Pr(Fi,t ⊂ N \ Na,t ) > 0 or (b) there are at players for which Pr(Fi,t ⊂ N \ Na,t ) > 0. The no directional bias assumptions restricts the analysis to psychological biases that increase the variance of someone’s beliefs without explicitly biasing that person towards a particular action. The false consensus effect, availability, and representativeness all overweight data that are independent of the strategy labels and payoffs of the game. In contrast, loss aversion could be modeled as players being more likely to overweight data that would produce lower-than-expected payoffs from the perspective of their current strategy, which might favor one strategy over the other. No directional bias rules out loss aversion. Aggregate randomness strengthens no directional bias by imposing that a large enough share of the population has biases which create support on fi,t = 0 and fi,t = 1. For r = 1 only, no directional bias implies aggregate randomness. Fi,t realizations can be thought of as forming a random network between players each period. Loosely, what is required for my results is that there is enough belief-based noise that as η increases, the population becomes more and more able to escape from an equilibrium via errors in belief. The aggregate randomness assumption rules out implausible pathological cases such as every player appearing in exactly r other players’ focal sets. The strategy updating function together with a belief formation process define a Markov process. In simple cases this process is on a state space A = {0, . . . , n} with state at .5 5
All of biases defined above operate on this state space. More complicated biases or combinations of biases may require a state space includes individual choices as well as aggregate strategy choices.
13
This Markov process is ergodic for > 0 and hence has a unique stationary distribution placing probability µ(a) on each state a, which the system converges to as t → ∞ from any initial state. An equilibrium is stochastically stable if for the state a corresponding to that equilibrium, µ∗ (a) ≡ lim µ(a) > 0. A basic result of the literature is that in 2×2 coordination →0
games with Bayesian players (i.e., pˆi,t = pt ), if A is risk-dominant, then µ∗ (n) = 1 and A is the unique stochastically stable equilibrium (Young 1993; Kandori et al. 1993). In many cases µ∗ can be completely determined by examining the properties of the unperturbed ( = 0) Markov process (Ellison 2000). A limit state of the unperturbed process is a state for which Pr(at+1 = at ) = 1, and I denote limit state a = i by ωi . The basin of attraction of a limit state ω, D(ω), is the set of states from which ω is eventually reached with probability 1 under the unperturbed dynamic. In addition, the properties of the unperturbed process often provide information about transition speeds. Let WA (n, ) be the expected wait time before the system reaches the basin of attraction of the all-A state, D(ωn ), if there are n players and the optimization error probability is , and let WB (n, ) be defined likewise.6
False consensus and availability in 2 × 2 coordination
4
games This section focuses on two biases for which focal observation sets are naturally singletons – the false consensus effect and the availability heuristic – both because they are of independent interest and as a means of providing intuition for my more general results. It is useful to think of player i’s focal observation as shifting p∗ , the minimum fraction of players playing 6
The appendix contains formal definitions and proofs. WA (n, ) is defined in terms of reaching the basin of attraction of a = n because once that basin of attraction is reached, there is a high probability that the system will be close to a = n for many periods after that.
14
BR(p ˆi,t ) = B if si,t = B
p 0
1 ∗
p
∗
p
p∗
1 2
BR(p ˆi,t ) = A if si,t = A D(ω0 )
D(ωn )
Figure 2: Best-Response Regions Under the False Consensus Effect. In shaded region, p drifts towards to 1/2; to left of shaded region, p jumps to close to 0; and to right of shaded region, p jumps to close to 1. A such that A is a best response: either reducing it if fi,t = 1 causes pˆi,t > p, or increasing it if fi,t = 0 causes pˆi,t < p. Let p∗ be the minimum pt such that BR(ˆ pi,t | fi,t = 1) = A and let p∗ be the minimum pt such that BR(ˆ pi,t | fi,t = 0) = A. These cutoffs are o n p∗ − η o n p∗ ∗ and p ≡ min ,1 . p ≡ max 0, 1−η 1−η ∗
(1)
They form an interval around p∗ within which the movement of the system is influenced by the realized focal observations as well as by optimization errors.
False consensus effect: Players with this bias place undue weight on their own strategy choice, producing a “solipsistic region” around p∗ where BR(ˆ pi,t | fi,t = 1) = A and BR(ˆ pi,t | fi,t = 0) = B. These states a ∈ np∗ , np∗ are never exited except through optimization errors, which cause the state to drift towards a = n/2. There are approximately (n − a) switches from si,t = B to si,t+1 = A each period and a switches from si,t = A to si,t+1 = B, so if a Q n/2, E[at+1 | at ] Q at . Outside of the solipsistic region standard myopic best response dynamics operate. Figure 2 depicts this adjusted dynamic. For η ≥ p∗ , the solipsistic region encompasses D(ω0 ), the entire basin of attraction of the 15
all-B state, and so the state drifts towards D(ωn ) due to optimization errors. If η < 1 − 2p∗ , p∗ < 1/2, so the system drifts into D(ωn ) rather than drifting to p = 1/2 and remaining there, which it would if p∗ < 1/2 < p∗ . For η ∈ [p∗ , 1 − 2p∗ ), numerical solution for WA (n, ) finds that WA (n, ) ≈ −1 for any n.7 For η ∈ / (p∗ , 1 − 2p∗ ), the fastest transition path from B to A combines (fast) linear increases in p in the solipsistic region with (slow) waiting for enough simultaneous errors to enter it from D(ω0 ) or to exit it into D(ωn ). For low biasedness η < p∗ , the fastest transition path from B to A involves first a jump from D(ω0 ) to p > p∗ , the point at which linear growth takes over, after which p grows quickly until it reaches the basin of attraction of A. For high biasedness, the fastest transition path begins with quick growth to p = 1/2 and then requires a jump from around p = 1/2 into D(ωn ). By replacing a large jump with a smaller jump plus linear growth, a population’s bias reduces WA (n, ), in the same manner as Norman (2009a). However, as n increases, the smaller jump still takes more and more time in expectation to occur, so lim WA (n, ) = ∞. n→∞
While WB (n, ) also falls as η increases, an increase in η causes p∗ to fall faster than p∗ rises, and the wait times are of order proportional to p∗ and 1 − p∗ respectively. However, if η > 1 − p∗ , then the solipsistic region encompasses the entire state space. When all players are solipsists in every state, the stationary distribution puts mass on all a ∈ A with greatest weights on states near a = n/2. This occurs because the stochastic process is completely symmetric with regards to strategy labels, due to the symmetry of optimization errors and BR(ˆ pi,t ) being independent of pt . Then as → 0, the stochastic process resembles a logistic birth-death process with µ ∼ B(n, 1/2). 7
I calculate expected wait times numerically through first-step analysis. See Appendix B for greater detail.
16
Availability heuristic: The false consensus effect bias with Fi,t = {i} is specific in generating solipsistic behavior, but it is generally true that errors in belief as well as optimization errors influence behavior in the region (p∗ , p∗ ). Players following the availability heuristic place too much weight on the same focal observation. Because F1t = · · · = Fnt , f1t = · · · = fnt , so all members of the population either simultaneously overestimate p or underestimate it. If the population overestimates p and p ≥ p∗ , then pˆi,t ≥ p∗ for all players, so all members of the population except those making optimization errors switch to A. The most likely transition path out of D(ω0 ) is for random optimization errors to lead to pt ≥ p∗ and then for the entire block of biased players to switch to A when the focal observation is A. Instead of changing part of the transition process to a fast linear one, the availability heuristic instead makes large numbers of simultaneous errors more likely. Because the errors in belief are not as strongly correlated period to period as with false consensus, revision errors have only weak serially correlation. Therefore the system never “gets stuck” around p = 1/2. (Belief errors are strongly correlated within a period.) The next section analyzes the more general setting in which members of the population have other biases than these examples. In these two examples the fastest transition routes to ωn are evident, but in the more general setting the corresponding transition routes are undetermined. However, Assumptions 1 and 2 guarantee that for small optimization error rates, the fastest transition to ωn relies on errors in belief. Generally these new possibilities lead to qualitatively faster transitions because they allow positive correlation between errors. However, errors in belief can be qualitatively slow instead, either because the correlations are too weak (e.g., if η < p∗ ) or too strong (as in the case of false consensus effect and η > 1 − 2p∗ ). Understanding these error in belief effects is easier in the large population limit, so after first considering the rare optimizatiom errors limit, I turn to the other limit.
17
5
General asymptotic theory
As is typical in the stochastic stability literature, I analyze the limiting behavior of the system as → 0 or n → ∞. I employ Ellison (2000)’s radius-coradius technique to establish the -limit results and a deterministic approximation of the evolutionary process as a difference equation for the n-limit results. The small- results apply to any mixture of focal-observation biases with no directional bias, while the large-n results depend on the exact bias or biases the population suffers from. For the small- theory I assume that η < 1 − p∗ for both substantive and technical reasons. Substantively, η ≥ 1 − p∗ implies η > 1/2. If biases were that extreme, then the evidence on cognitive biases would be much more conclusive than it is. Technically, when η is slightly higher that 1 − p∗ , the payoff advantage of strategy A is too small to overcome the additional noise introduced by the bias, so that the the A equilibrium is no longer the unique long-run outcome.8 Finally for small n, integer problems cause the risk-dominant equilibrium to r(1−η) 1−η not be stable, so I assume the population size is n > n ≡ max{ 1−p ∗ −η , 1−2p∗ }. Provided
these conditions are satisfied, the long-run outcome in a population with heterogenous focalobservation biases that have no directional bias and that satisfy aggregate randomness is the same as that of of a Bayesian population. Proposition 1. For 2×2 coordination games played by a biased population, the risk-dominant strategy is uniquely stochastically stable. In addition transitions between equilibria are faster in biased populations:9 8
I discuss this issue, which is caused by integer problems combined with uniform optimization errors, further following my proof of Proposition 1. 9 Recall that dxe denotes the smallest integer greater than x. In the following proposition, because at is an integer, conditions like p > p∗ become a > dp∗ ne after accounting for the state space.
18
Proposition 2. For 2 × 2 coordination games played by a biased population, then for low enough , i) there is a cA > 0 such that the expected wait time to reach the risk-dominant equilibrium WA (n, ) < cA −dp
∗ ne
) if η < p∗ or WA (n, ) < cA −r otherwise; and
ii) there is a cB > 0 such that the expected wait time to leave the risk-dominant equilibrium WB (n, ) > cB −d(1−p
∗ )ne
.
Biases increase transition speeds because they cause correlation in errors. With Bayesian populations all states within a basin of attraction share the same best response, so players’ revision errors are not correlated between players during a period or over time. Because movement between basins of attraction requires sufficiently many simultaneous errors, uncorrelated errors cause the population to converge slowly to the long-run distribution. Errors in belief lead to behavior that depends the exact state pt rather than on the which basin of attraction the state is in, so that revision errors are positively correlated in the region a ∈ (p∗ n, p∗ n). For lower η, the evolutionary system blends these dynamics and the uncorrelated dynamic, making exits from both D(ω0 ) and D(ωn ) easier. Figure 3 shows calibrated WA (N, ) as η varies for homogenous populations with several biases. The curve labeled (FCE) & (RH), r = 2 is a population where each player overweights themselves and a random other player. For η > p∗ , WA (N, ) ≈ r , although the wait times begin to increase for the representativeness heuristic and false consensus effect with r = 1 for η > 1 − 2p∗ . The transition speeds in Proposition 2 are for fixed n. Even when WA (n, ) is bounded above by cA −r , the expected wait time can still increase sharply with population size if cA increases with n, so next I consider the conditions necessary for there to be “small” wait times even for large population sizes. The exact form of focal-observation bias determines 19
107 Availability, r = 1 Representativeness Heuristic, r = 1 False Consensus, r = 1 WA (n, )
10
5
(FCE) & (RH), r = 2 Availability, r = 2
103
101 0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
η
Figure 3: Speeds of Transition to Risk-Dominant Equilibrium by Bias for r = 1, p∗ = 1/3, n = 50, and = 1/12. With Bayesians (η = 0) the transition speed WA (50, 1/12) ≈ 6 million periods, while for r = 1 biases with η > 2/10, WA (50, 1/12) < 150 periods. In contrast, WB (50, 1/12) > 6 million periods for any η < 6/10. how cA is related to n, so I consider homogenous populations sharing one of the three biases described in Section 2. For these propositions, let WA () ≡ lim sup WA (n, ) be the upper bound on the expected n→∞
wait time to reach D(ωA ). If WA () < ∞, I say the system displays “fast convergence” to the A equilibrium (Kreindler and Young 2013; Ellison, Fudenberg and Imhof 2014). Let x(p) ≡ E[pt+1 |pt = p] denote the expected share of A players in the next period if this period fraction p played A. The idea behind the following propositions is that for populations with serial-correlation-producing biases and large n, the realized share of A players pt+1 (pt ) ≈ x(pt ). If outside of D(ωn ), x(pt ) − pt > δ > 0, then D(ωn ) will be reached in finite time regardless of n. See Sandholm (2001), Benaim and Weibull (2003), or Sandholm (2010) for more
20
general discussion of treating the large-population limits of stochastic evolutionary processes as deterministic evolutionary processes. Unlike the small- case, the system spends most of the time not at a = 0 or a = n but around a = n or a = (1 − )n. With non-vanishing errors, a slightly smaller η is enough for the system to escape D(ω0 ) easily because that requires p∗ < instead of p∗ = 0. The former condition is equivalent to η ≥
p∗ − . 1−
A similar adjustment (p∗ < 1 − ) must be made
so that D(ωn ) contains the noisy state when it is close to ωn . Proposition 3. For 2 × 2 coordination games played by a population with a focal-observation belief-formation process with sample size r = 1, fast convergence to A occurs if and only if i) (for (RH) or (FCE) biases) ii) (for (AH) biases)
p∗ − 1−
p∗ − 1−
≤η<
≤ η < 1 − 2p∗ , or
1−p∗ − . 1−
With r = 1, numerical calculations find that WA () ≈ −1 if the above conditions hold, so transitions to A are also fast in practical terms for intermediate η. For r > 1, some additional conditions are needed for fast convergence. Like the false consensus effect, the representativeness heuristic produces serial correlation in error rates in states a ∈ (np∗ , np∗ ). With r > 1, fi,t has support on values other than 0 and 1, and the strength of the serial correlation varies with exactly how many s = A focal observations are necessary to produce it. For each population share of A-players p, there is a minimum share of s = A observations in a focal observation set, f (p), for which if fi ≥ f (p), then BR(ˆ pi,t ) = A. f (p) is f (p) = min f ∈ {0, 1/r, . . . , 1} ∪ {n} : ηf + (1 − η)p ≥ p∗ ,
21
(2)
where I adopt the convention that f (p) = n if BR(ˆ pi,t ) = B even when fi,t = 1. Then
x(pt ) = + (1 − 2) Pr(fi,t ≥ f (pt )).
(3)
Typically f (p∗ ) = 1 – every member of Fi,t must be playing A at t in order to guarantee pˆi,t ≥ p∗ , but as p increases f (p) falls until f (p∗ ) = 0. For example, under the representativess heuristic with r = 2, for p > p∗ , f (p) = 0, while for p < p∗ , even with Fi,t ⊂ NA,t , BR(ˆ pi,t ) = B, so f (p) = n. But in the region between these cutoffs, whether there are 1 or 2 focal observations that are s = A matters. BR(ˆ pi,t |fi,t ≥ 1) = A if η/2 + (1 − η)p ≥ p∗ , or p ≥
f (p) =
n if p < p∗ 1 if p ∈ [p∗ , p∗0 ) 1 2 0
and
x(p) =
∗0
2p∗ −η , 2(1−η)
so
+ (1 − 2)p2
if p < p∗ 0
if p ∈ [p∗ , p∗ ) .
0 + (1 − 2)(2p − p2 ) if p ∈ [p∗ , p∗ ) 1 − if p > p∗
if p ∈ [p , p∗ ) if p > p∗
Proposition 4. In 2 × 2 coordination games played by a population with the (RH) beliefformation process with sample size r > 1, fast convergence to A occurs if and only if (i) the population is sufficiently biased (η ≥ p∗ ), and (ii) the population has a high enough optimization error rate (for some (r, η) ∈ [0, 1/2), > (r, η)), and (iii) if r is odd, the population is not too biased (η ≤ r(1 − 2p∗ )). Proposition 4 enumerates three possible causes for slow convergence to the A equilibrium 22
1
1 1−
r=4
r=1
E(pt+1 )
E(pt+1 )
1−
r=3
r=1 r=3 r=4
0 0
p∗
p∗
1
p∗
0 0 = p∗
p∗
p∗
pt
1
pt
∗
∗
4(a): Dynamics with η < p and r = 1. In this 4(b): Dynamics with η > p and varying number of case, as n grows large, the system remains near p = focal observations. For r = 1 (solid blue line), the system remains near p = 1/2 indefinitely. For r = indefinitely. Here = 0.05, η = 0.2, and p∗ = 0.3. 3 (dotted brown line), the system remains near the marked point indefinitely. For r = 4 (dashed red line) pt grows linearly and there is fast convergence to A. In all cases = 0.125, η = 0.5, and p∗ = 0.3.
Figure 4: Dynamics under Representativeness Heuristic for Large Population Size n. In the region (p∗ , p∗ ) surrounding p∗ , there is positive feedback, sometimes reducing the expected time in B, WA (N, ). The distribution of fi,t conditional on pt influences the shape of E[pt+1 ], and if for all p in [0, p∗ ], E[pt+1 | pt ] > pt , then there is fast convergence to A.
23
under the representativeness heuristic for large n, all of which are illustrated in Figure 4. First, the bias could be too small. If p∗ > 0 because η < p∗ , as in Figure 4(a), then as n → ∞ the system remains near p = for arbitrarily long times if the population starts in the B equilibrium. A second reason the system can become fail to leave D(ω0 ) quickly is shown by the blue curve in Figure 4(b). For r = 1, x(1/2) = 1/2, so if n/2 ∈ D(ω0 ), p stays at p = 1/2, so the system takes arbitrarily long to reach D(ωn ) as n becomes large. For r = 1 this occurs if η > 1 − 2p∗ but this phenomenon can occur with every odd r > 1 if at p = 1/2, half of the Fi,t must be playing A for i to switch to A.10 Condition (iii) of Proposition 4 guarantees that at p = 1/2, Pr(fi ≥ f (p)) = x(p) > 1/2. If players’ focal observation set sizes varied, then this condition would be unnecessary. The final reason that wait times to reach ωn can grow large for large n is if positive feedback in (p∗ , p∗ ) is not strong enough to produce E[pt+1 | pt ] > pt for every pt ∈ / D(ωn ). In the r = 3 case in Figure 4(b), there is a small interval when p is slightly below p∗ in which E[pt+1 | pt ] < pt . As n grows large the system spends an arbitarily long time near the first root of x(p) − p, the point marked in the figure, instead of reaching D(ωn ). Generally, however, with high enough , optimization errors can substitute for the weak positive feedback of errors in belief, which condition (ii) captures. Intuitively, the geometric effect of an increase in is that the E[pt+1 | pt ] curve rotates clockwise.11 Loosely, the fast convergence literature is based on observations akin to mine that if outside of D(ωn ), x(pt ) − pt > δ > 0, then D(ωn ) will be reached in finite time regardless of n. 10
If f (1/2) = (r + 1)/2 then x(1/2) = 1/2 because Pr(fi ≥ (r + 1)/2) = Pr(fi < (r + 1)/2), and p = 1/2 is a steady state of the difference equation pt+1 = x(pt ). 11 The exact cutoff is sensitive to the particular form that optimization errors take and has no closed-form solution given the uniform trembles specification. Kreindler and Young (2013) and Ellison, Fudenberg and Imhof (2014) analyze similar evolutionary systems with logit errors, under which fast convergence typically occurs for a larger range of optimization noise levels than my trembles specification.
24
The particular biases in my model guarantee that the rate of increase of pt has a positive δ lower bound, while in Kreindler and Young (2013) and Ellison et al. (2014) the lower bound is produced by the structure of the logit error function (although Ellison et al. (2014) do consider more general processes briefly). Ellison et al. (2014) establish some limitations on fast convergence for logit best-responses reminscent of Proposition 4. Finally I show that Propositions 1 and 2 hold true for larger games as well by extending Ellison (2000)’s general technique to biased populations. I relegate the exact details to the appendix, and focus on the qualitative effects of biases. I now consider a population recurrently playing a symmetric, weakly acyclic m × m game with multiple pure-strategy Nash equilibria.12 With m strategies, the state space is an m-dimensional simplex Θ = {θ ∈ N m : θ1 + · · · + θm = n} and ωi denotes the limit state in which θi = n. The basin of attraction for any ωi becomes all states θ for which BR(ˆ pi,t ) = si for any feasible fi,t given θ. If the fastest transitions between limit sets for η = 0 are movement along a direct path between the limit sets on the boundary of the simplex, the dynamics of these transitions resembles those in the 2-strategy case. If one ignoring the existence of other strategies, the transition from ωi to ωj behaves in exactly the same way that transitions do in 2 × 2 games. Let cη (ωi , ωj ) be the share of the population that must make optimization errors in order to reach ωj from ωi . Taking these two strategies in isolation, cη (ωi , ωj ) = analogous to p∗ = cη (ωB , ωA ) =
p∗ −η 1−η
and p∗ = 1 − cη (ωA , ωB ) =
1−p∗ −η 1−η
c0 (ωi ,ωj )−η , 1−η
=
p∗ . 1−η
exactly
With more
than two strategies, transitions from ωi to ωj may involve focal observation sets containing strategies other than sj , but if there is a focal observation set more favorable to ωj than observing players using sj , this can only cause the speed of transition to decrease at a faster 12
A weakly acyclic game is one in which, under the unperturbed dynamic, for every state there is some positive-probability path of best replies to a pure strategy limit state of the game. This assumption intentionally rules out stable mixed-strategy Nash equilibria, because mixed-strategy equilibria are problematic given the assumption that the entire population updates.
25
rate. In addition, in some cases the fastest transition from ωi to ωj is for several si players to switch to sk 6= sj if sj is a best response to sk . The availability of more efficient focal sets means that with more than 2 strategies, the cost in errors to move between ωi and ωj falls in η at least as quickly as the case with two strategies, and often more quickly (Lemma A5). It has long been known that in m-strategy games, sometimes the fastest transition between limit states will be an indirect transition (Young 1993). For example, a direct transition from ωB to ωA might require fewer errors than an indirect path that goes from ωB to ωC and finally to ωA but the indirect path could be faster because the errors are divided into two smaller jumps, each of which has higher probability than the direct jump (see Ellison for further intuition). Interestingly, for multi-step transitions the order of the expected wait time often decreases faster in η than for single-step transitions. Loosely, the order of each step decreases at least at rate η/(1 − η), and the full transition’s wait time is of order less than the sum of the orders of each step (Lemma A8). Therefore with more than two strategies, speeds to reach a particular limit set in many instances increase more quickly in η than with two strategies. In contrast, the speed of exiting each limit set decreases at exactly the same rate as in the two-strategy case. For any η, the fastest route out of D(ωi ) always involves a direct transition to a fixed ωj along the i − j boundary. Because sj is the best route of escaping from ωi , the fastest escape also involves players having focal observation sets containing entirely players using sj . Therefore sj and the payoffs of si and sj combinations entirely determine R(ωi ) (Lemma A7). Most fastest transition paths are on the boundary of the simplex, and the dynamics of these transitions resembles those in the earlier 2-strategy case. However, there are rare cases in which fastest transition from ωi to ωj is not a boundary transition but instead a jump to the a mixed strategy in the interior of the simplex, for example with θi , θj , θk > 0. With
26
interior transitions, as η increases from η = 0, the number of errors required initially follows the same pattern as boundary transitions. Both sj and sk errors are required to move out of ωi , and for low η if players’ focal observation sets place full weight on either sj or sk , the overweighting reduces the number of errors necessary by the same amount as for a boundary transition. However, as η increases further, belief errors alone cannot simultaneously produce sj and sk revision errors, and so more optimization errors are required than in a boundary transition, where optimization errors can be fully replaced by belief errors. I write Wi← (n, ) for the maximum expected wait time to reach ωi from anywhere outside of D(ωi ) and Wi→ (n, ) for the minimum expected wait time to leave D(ωi ). The expected wait time for a transitions between limit states is polynomial in the minimum number of errors necessary to produce the transition. For a given ωi , I will define p∗i (η) (analogous to p∗ in the 2 × 2 case) based on p∗i , the fraction of strategy errors necessary to reach ωi from any other limit state, and qi∗ (η) (which is analogous to 1 − p∗ ) based on qi∗ , the fraction of strategy errors necessary to leave ωi and reach any other limit state. Proposition 5. If ωi is a limit point in a weakly acyclic m × m symmetric game, then for p∗i (η) ≡
p∗i −η 1−η
and qi∗ (η) ≡
qi∗ −η , 1−η
i) there is a ci→ > 0 such that the expected wait time to leave ωi is Wi→ (n, ) > ∗
ci→ −dqi (η)ne . Furthermore, if at η = 0 the most expensive minimum-error path from any other ω 0 to ω uses only boundary transitions, ∗
ii) there is a ci← > 0 such that the expected wait time to reach ω is Wi← (n, ) < ci← dpi (η)ne if p∗i (η) > 0 or Wi (n, ) < ci← r if p∗i (η) ≤ 0.
27
An immediate corollary is that if ω satisfies the conditions of the proposition and q ∗ (0)∗ > p∗ (0), then for all η ∈ [0, q ∗ (0)), ω is stochastically stable. This in turn implies that 12 dominant equilibria in m-strategy games are stable in biased populations, generalizing Ellison (2000), Corollary 1.13 Ellison’s comparison of errors technique, used in Proposition 1 to establish stochastic stability, is a sufficient condition which does not universally apply. For some m-strategy games, there is no ω for which q ∗ > p∗ . However, Ellison’s technique for bounding convergence speeds is more general than this test for stochastic stability, and so even in games where stochastic stability is indeterminate using Proposition 5, the proposition does establish that biases make evolution faster. In addition, while my analysis is for two-player weakly acyclic games, Ellison’s technique applies to the more general class of games in Ellison (2000), Definition 1. Hence, it seems that my speedup results would also apply in contexts where the radius-coradius technique “fails”, such as Neary (2012). One might worry that Proposition 5 does not apply to some transitions. However, interior transitions do still become faster as η increases, albeit at a slower rate, which potentially could change the long-run equilibrium for high enough η. Restricting the minimum-error route to ω to be a boundary transitions is a sufficient condition for the proposition, not a necessary one. If the η = 0 stochastically stable equilibrium is sufficiently stable (in the sense that p∗ is sufficiently smaller than q ∗ ) then interior transitions, while potentially decreasing in speed at lower rates, are still fast enough to not affect what the stochastically stable outcome is as η changes. In addition, in many games interior transitions are never minimum-error paths, in which case the condition is moot. Finally, even if an interior transition is needed to construct the slowest path to ω, the above bound on W¬s (n, ) holds initially for η < 0. 13
Morris et al. (1995) define an equilibrium (s, s) as p-dominant if s is a best response to any mixed strategy that plays s with p or greater probability.
28
An alternate non-behavioral interpretation of my model is that focal observation sets represent ephemeral local interactions. If the connections in Fi,t are symmetric (j ∈ Fjt ⇔ i ∈ Fjt ), then focal sets can be thought of as a set of r connections in a random network. The network is not fixed for all time, but instead is regenerated randomly with probability 1 − η every period. Then the beliefs players use to revise their strategies accurately reflect their expected interactions next period: with probability η they interact with their current neighbors, and with probability 1 − η interact with new neighbors. Ellison (1993)’s 2k nearest neighbors model would be a special case with η = 1 and Fi,t = {i − k, . . . , i − 1, i + 1, . . . , i + k} and r = 2k (see also Ellison 2000, Section 5, or L´opez-Pintado 2006). Under this interpretation, my results suggest that evolution on a relatively unstable network still behaves like evolution on a fully stable network. However, care needs to be be taken because my aggregate randomness assumption rules out some network structures. For instance, Bala and Goyal (1998) shows that the presence of a a small set of agents who are connected to everyone can generate lock-in on an inferior technology in a social learning model. Ellison, Fudenberg and Imhof (2014) explores in greater detail the connections between sampling and network models.
6
Conclusion
Behavioral game theorists have produced several static equilibrium refinement models based on psychological considerations. This paper advances a classic equilibrium refinement based on evolutionary / dynamic considerations and shows it is applicable for a variety of biases. Ironically, the long-run evolutionary outcome for a Bayesian population is a better prediction of behavior for populations with moderate probability judgement biases than for Bayesian populations, since the identical outcome is reached more quickly by a biased population. 29
Under best response dynamics the limiting factor on the speed of the convergence is that achieving enough simultaneous errors to switch equilibria is a rare event. Biases accelerate behavioral evolution because they introduce positive correlation in errors across players or over time, which increases the likelihood of these events. If biases do not explicitly favor one strategy, the equilibrium which is stochastically stable is independent of the degree of bias because all transitions increase in speed proportionally. It would be interesting to extend the m-strategy analysis to games with more structure, such as extensive form games or games with player types, or to consider evolution in a multi-population setting where sub-populations have differing levels of bias. In addition, connections between focal-observation belief-formation and network or sampling models could be explored further with a focal-observation bias-like framework.
30
References Alicke, Mark D. and Edward Largo, “The Role of the Self in the False Consensus Effect,” Journal of Experimental Social Psychology, 1995, 31, 28–47. Bala, Venkatesh and Sanjeev Goyal, “Learning from Neighbors,” Review of Economic Studies, 1998, 65, 595–621. Barber, Brad M., Terrance Odean, and Ning Zhu, “Systematic Noise,” Journal of Financial Markets, 2009, 12, 547–569. Barberis, Nicholas, Andrei Shleifer, and Robert Vishny, “A Model of Investor Sentiment,” Journal of Financial Economics, 1998, 49, 307–343. Benaim, Michel and Jorgen W. Weibull, “Deterministic Approximation of Stochastic Evolution in Games,” Econometrica, 2003, pp. 873–903. Binmore, Ken and Larry Samuelson, “Muddling Through: Noisy Equilibrium Selection,” Journal of Economic Theory, 1997, 74, 235–265. Blanco, Mariana, Dirk Engelmann, Alexander K. Koch, and Hans Theo Normann, “Preferences and Beliefs in a Sequential Social Dilemma: A Within-Subjects Analysis,” 2014. Blume, Lawrence E., “How Noise Matters,” Games and Economic Behavior, 2003, 44, 251–271. Costa-Gomez, Miguel, Vincent P. Crawford, and Bruno Brosetta, “Cognition and Behavior in Normal-Form Games: An Experimental Study,” Econometrica, 9 2001, 69 (5), 1193. Dawes, Robin M., “Statistical Criteria for Establishing a Truly False Consensus Effect,” Journal of Experimental Social Psychology, 1989, 25, 1–17. Ellingsen, Tore, Magnus Johannesson, Sigve Tjøtta, and Gaute Torsvik, “Testing guilt aversion,” Games and Economic Behavior, 2010, 68, 95–107. Ellison, Glenn, “Learning, Local Interaction, and Coordination,” Econometrica, 1993, 61, 1047–1071. , “Basins of Attraction, Long-Run Stochastic Stability, and the Speed of Step-by-Step Evolution,” Review of Economic Studies, 2000, 67, 17–45. , Drew Fudenberg, and Lorens A. Imhof, “Fast convergence in evolutionary models: A Lyapunov approach,” 2014. Unpublished manuscript. 31
Engelmann, Dirk and Martin Strobel, “The False Consensus Effect Disappears if Representative Information and Monetary Incentives Are Given,” Experimental Economics, 2000, pp. 241–260. and , “Deconstruction and reconstruction of an anomaly,” Games and Economic Behavior, 2012, pp. 678–689. Foster, Dean and H. Peyton Young, “Stochastic Evolutionary Game Dynamics,” Theoretical Population Biology, 1990, 38, 219–232. Freidlin, M.I. and A.D. Wentzell, Random Perturbations of Dynamical Systems, New York: Springer-Verlag, 1998. G¨ achter, Simon, Daniele Nosenzo, Elke Renner, and Martin Sefton, “Who Makes a Good Leader? Cooperativeness, Optimism, and Leading-by-Example,” Economic Inquiry, 2012, 50, 953–967. Harsanyi, John C. and Reinhard Selten, A General Theory of Equilibrium Selection in Games, Cambridge: MIT Press, 1988. Heller, Yuval, “Three Steps Ahead,” Theoretical Economics, forthcoming. Kahneman, Daniel, Thinking, Fast and Slow, New York: Farrar, Straus and Giroux, 2011. Kandori, Michihiro, George J. Mailath, and Rafael Rob, “Learning, Mutation, and Long Run Equilibria in Games,” Econometrica, 1993, 61 (1), 29–56. Khan, Abhimanyu and Ronald Peeters, “Cognitive hierarchies in adaptive play,” International Journal of Game Theory, forthcoming. Kreindler, Gabriel E. and H. Peyton Young, “Fast Convergence in Evolutionary Equilibrium Selection,” Games and Economic Behavior, 2013, 80, 39–67. Kuran, Timur and Cass R. Sunstein, “Availability cascades and risk regulation,” Stanford Law Review, 1999, pp. 683–768. L´ opez-Pintado, Dunia, “Contagion and coordination in random networks,” International Journal of Game Theory, 2006, 34, 371–381. Maes, Michael and Heinrich H. Nax, “A Behavioral Study of ’Noise’ in Coordination Games,” 2014. Unpublished manuscript. Marks, Gary and Norman Miller, “Ten years of research on the false-consensus effect: An empirical and theoretical review,” Psychological Bulletin, 1987.
32
Maruta, Toshimasa, “Binary Games with State Dependent Stochastic Choice,” Journal of Economic Theory, 2002, 103, 351–376. Mohlin, Erik, “Evolution of theories of mind,” Games and Economic Behavior, 2012, 75, 299–318. Montanari, Andrea and Amin Saberi, “The spread of innovations in social networks,” Proceedings of the National Academy of Sciences, 2010, 107 (47), 20196–20201. Morris, Stephen, Rafael Rob, and Hyun Song Shin, “p-Dominance and Belief Potential,” Econometrica, 1995, pp. 145–157. Neary, Philip R., “Competing conventions,” Games and Economic Behavior, 2012, 76, 301–328. Norman, Thomas W.L., “Rapid Evolution Under Inertia,” Games and Economic Behavior, 2009, 66, 865–879. , “Skew-Symmetry Under Simultaneous Revisions,” International Game Theory Review, 2009, 11, 471–478. Osborne, Martin J. and Ariel Rubinstein, “Games with Procedurally Rational Players,” American Economic Review, 1998, 88, 834–847. Oyama, Daisuke, William H. Sandholm, and Olivier Tercieux, “Sampling Best Response Dynamics and Deterministic Equilibrium Selection,” Theoretical Economics, forthcoming. Proto, Eugenio and Daniel Sgroi, “Self-Centered Beliefs on Individuals’ Choices and Characteristics,” 2013. Working paper. Rabin, Matthew, “Inference by Believers in the Law of Small Numbers,” Quarterly Journal of Economics, 2002, 117 (3), 775–816. and Dimitri Vayanos, “The Gamblers and Hot-Hand Fallacies: Theory and Applications,” Review of Economic Studies, 2010, 77, 730–778. Ross, Lee, “The false consensus effect: An egocentric bias in social perception and attribution processes,” Journal of Experimental Social Psychology, 1977. Sandholm, William H., “Almost global convergence to p-dominant equilibrium,” International Journal of Game Theory, 2001, pp. 107–116. , Population Games and Evolutionary Dynamics, MIT Press, 2010.
33
Sawa, Ryoji, “An Analysis of Stochastic Stability in Bargaining Games with Behavioral Agents,” 2012. Unpublished manuscript. Schwarz, Norbert, Herbert Bless, Fritz Strack et al., “Ease of Retrieval as Information: Another Look at the Availability Heuristic,” Journal of Personality and Social Psychology, 1991, 61, 195–202. Sethi, Rajiv, “Stability of Equilibria in Games with Procedurally Rational Players,” Games and Economic Behavior, 2000, 32, 85–104. Stahl, Dale O. and Paul W. Wilson, “Experimental Evidence on Players’ Models of Other Players,” Journal of Economic Behavior and Organization, December 1994, 25 (3), 309–327. Tversky, Amos and Daniel Kahneman, “Belief in the Law of Small Numbers,” Psychological Bulletin, 1971, 76, 105–110. and , “Judgement Under Uncertainty: Heuristics and Biases,” Science, 1974, 185, 1124–1131. Vanberg, Christoph, “A short note on the rationality of the false consensus effect,” 2008. unpublished manuscript. , “Why Do People Keep Their Promises? An Experimental Test of Two Explanations,” Econometrica, 2008, 76, 1467–1480. Young, H. Peyton, “The Evolution of Conventions,” Econometrica, 1993, 61 (1), 57–84.
34
A
Proofs
Several proofs use the radius-coradius technique of Ellison (2000), which bounds transition speeds by comparing how difficult it is for the system to move between different sets of states defined by the unperturbed process, the Markov process for parameter = 0 (but note η > 0 is assumed under the unperturbed process). Let c(x, y) be the minimum number of errors necessary to reach state a = y from state a = x and ω be a limit state of the unperturbed process given by some belief-formation function defined in Section 3 with η ≥ 0 and optimization error rate = 0. Then the radius of ω, R(ω) ≡ mina∈D(ω) c(ω, a), is the minimum cost to leave D(ω). and the coradius of ω, / CR(ω) ≡ maxa∈A c(a, ω) is the cost of reaching ω from the most costly state a ∈ A. Finally, consider a path from x to y that passes through intermediate limit states l(1), l(2), . . . , l(r) ∈ P {x, . . . , y}. Then the modified cost of that path is c∗ (x, y) = c(x, y) − r−1 i=2 R(l(i)) and the modified coradius is CR∗ (ω) = maxa∈A c∗ (a, ω).. These modified costs take into account that a series of small jumps is faster than a single large jump. The following lemma restates Ellison’s main radius-coradius result (Ellison (2000), Theorem 2). Lemma A1. Suppose that limit state ω is such that R(ω) > CR∗ (ω). Then i) ω is the long-run stochastically stable state of the system, and ∗ (ω)
ii) ∀a 6= ω, the expected wait to reach ω from a is O(− CR
) as → 0.
Two wait times of interest are the expected wait time before the system reaches either the basin of attraction D(ω0 ) or D(ωn ) from the state a = n or a = 0 respectively. Let WA (n, ) ≡ E[min{t | at ∈ D(ωn ), a0 = 0}] denote the expected wait time to reach the A basin of attraction and and let WB (n, ) be defined accordingly.
35
A.1
Proof of Proposition 1
First, the set of limit states is ω0 , ωn , and possibly some or all of the intermediate states ωa such that a ∈ (np∗ , np∗ ). Transitions for intermediate limit states under the unperturbed dynamic are entirely determined by each player’s focal observation set Fi,t . These intermediate limit states, if they exist, are relatively easy to escape: if ωk is an intermediate limit state, then in follows directly from Assumption 2 that R(ωk ) = 1. Now, consider c∗ (a, n) for any a ∈ A. The cost of intermediate limit states between a + 1 and n is both added and subtracted from c∗ (a, n), so c∗ (a, n) = R(a). Because intermediate limit sets are easy to escape, maxa c∗ (a, n) = c∗ (0, n), so CR∗ (ωn ) = R(ω0 ). All states such that with fi,t = 1, pˆi,t > p∗ are not in D(ω0 ), and all states with fi,t = 1, pˆi,t < p∗ are in D(ω0 ). The argument is by induction. At state a = n − 1, by Assumption 2, there is a positive probability of transitioning to limit state ωn . Now if state a has a positive probability of reaching ωn , state a−1 does as well if pˆi,t < p∗ with fi,t = 1. From Assumption 2, at least a players have probability greater than 0 of having pˆi,t > p∗ , so reaching state a from a − 1 has positive chance. Similar arguments establish that the states in which if fi,t = 0, pˆi,t < p∗ are not in D(ωn ), and all states in which if fi,t = 0, pˆi,t > p∗ are in D(ωn ). Now consider two cases: first, R(ω0 ) > r (if p∗ is sufficiently larger than η); second, R(ω0 ) = r (if p∗ ≤ η). In the first case, states a ≥ np∗ /(1 − η) are still in D(ωn ), so R(ωn ) ≥ dn − np∗ /(1 − η)e = dn(1 − p∗ )e. Now to bound R(ω0 ) > r if R(ω0 ) > r consider a ¯ ≡ dn(p∗ − η)/(1 − η)e. Either a ¯ ≥ r of the other case applies, so Assumption 2 applies and at a = a ¯,
pˆi,t (si,t = A | fi,t
a ¯ = 1) = η + (1 − η) ≥ η + n
36
1−η n
n(p∗ − η) (1 − η)
≥ p∗
so a ¯ ∈ / D(ω0 ). Therefore R(ω0 ) ≤ dn(p∗ − η)/(1 − η)e = dnp∗ e. Comparing R(ωn ) and CR∗ (ωn ), R(ωn ) − CR∗ (ωn ) ≥ dn(1 − p∗ )e − dnp∗ e ≥
n(1 − p∗ − η) n(p∗ − η) n(1 − 2p∗ ) − −1= −1 1−η 1−η 1−η
so R(ωn ) > CR∗ (ωn ) if n > n ≥
1−η . 1−2p∗
Alternatively, R(ω0 ) = CR∗ (ωn ) ≤ r. Then
R(ωn ) ≥ n(1 − p∗ − η)/(1 − η) > r provided n > n ≥
r(1−η) . 1−p∗ −η
In either case, ωn is uniquely
stable from Lemma A1. The assumption η < 1 − p∗ is necessary for r = 1 because otherwise R(ωn ) = R(ω0 ). If there are no intermediate limit states, µ∗ (ω0 ) = µ∗ (ωn ) = 1/2. In the case of r > 1, there are non-connected intervals of η in [1 − p∗ , 1] for which R(ωn ) = R(ω0 ) and similar results occur, although in general with r > 1 even in these intervals µ∗ (ω0 ) < 1/2 < µ∗ (ωn ). This behavior is exacerbated by the uniform optimization errors assumption, which makes the evolutionary process simple to analyze but produces behavior driven by integer problems for η > 1 − p∗ .
A.2
Proof of Proposition 2
For WA (n, ), the claim is an immediate consequence of Lemma A1’s claim (ii ), applying the same logic as the proof of Proposition 1 above. For WB (n, ), an analogous calculation to that claim’s is valid for any limit state (Ellison (2000), Lemma 6).
A.3
Proof of Proposition 3
I first prove that for large n , a deterministic approximation holds. Lemma A2. If Var(at ) is O(n), then lim pt+1 (pt ) = E[pt+1 |pt ]. n→∞
37
Proof. Var(at ) < cn so Var(pt ) < c/n. From Chebyshev’s inequality, for any e,
Pr(|pt+1 − E[pt+1 |pt ]| > e) <
Var(pt ) c = 2 2 e ne
which approaches zero as n → ∞. Now I separate the proposition’s claims into several parts. p) = B. For all biases and η < p∗ : in an interval around ω0 , all pˆ’s are such that BR(ˆ Hence from any at < a ¯ ≡ min{a | a 6= D(ω0 )}, at+1 ∼ B(n, ). From application of Chebyshev’s inequality, limn→∞ Pr(at+1 > a ¯) = 0. Because WA (n, ) ≥ E[min{t | at+1 > a ¯}] = Pr(at+1 > a ¯)−1 , limn→∞ WA (n, ) = ∞, so for this case WA () does not exist. Availability heuristic for η ≥ p∗ : in this high biasedness case, pˆi,t > p∗ if shared focal strategy observation had s = A, and then ∀i, BR(ˆ pi,t ) = A. The probability of this event is at least and WA () ≤ −1 . False consensus effect for η ≥ p∗ : For the false consensus effect and representativeness heuristic, I apply Lemma A2 and work with a deterministic difference equation approximation of population behavior. Consider at+1 . Let aat+1 ∼ B(at , 1 − ) be the number of players with st = A who have st+1 = A as well, and abt+1 ∼ B(n−at , ) be the number of players with st = B who have st+1 = A. Then at+1 = aat+1 +abt+1 and Var(at+1 ) = Var(aat+1 )+Var(abt+1 ) = n(1−), so the lemma’s conditions are satisfied. Now applying Lemma A2, if at t = 0, p0 = 0, then p1 = , p2 = (1 − ) + (1 − ), and in general pt = (1 − pt−1 ) + (1 − )pt−1 . Alternately, pt = 12 (1 − (1 − 2)t ). For any n the system reaches D(ωn ) in finite time as long as 1 − 2p∗ > η > p∗ : taking logs of p∗ < 12 (1 − (1 − 2)t ), for t >
ln(1−2p∗ ) , ln(1−2)
pt > p∗ . To show that WA (n, ) is unbounded for 1 − 2p∗ ≤ η, assume that
some WA () exists. But then for n large enough, at t = WA (), p = 12 (1 − (1 − 2)WA () ) < 1/2,
38
a contradiction. Representativeness heuristic for η ≥ p∗ : consider at+1 . This a binomial random variable constructed out of n Bernoulli trials with probability of success Pr(si,t+1 = A | pt ), i.e., at+1 ∼ B(n, + pt (1 − 2)). Since E[pt+1 | pt ] = + (1 − 2)pt , the analysis for the false consensus effect applies to this case as well.
A.4
Proof of Proposition 4
Let g(p) ≡ Pr(fi ≥ f (p)). As in the proof of Proposition 3, I show that a deterministic difference equation, pt+1 = x(pt ) = + (1 − 2)g(pt ), approximates the evolutionary process well for large n and then work with the difference equation. Figure 4(b) depicts some particular cases of x(p) and g(p). The following characteristic of g(p) is used in Proposition 4’s proof: Lemma A3. There is a point p¯ ∈ [0, 1/2) such that g(¯ p) = p and g(p) > p for p > p¯. Proof. g(p) = 1 − F (f (p), r, p) where F (z, r, p) is the binomial cumulative distribution function, so g(p) is composed of segments of succesive binomial CDF complements. It is a strictly increasing function with jumps upward at p where af (p) is discontinuous. Furthermore, it is s-shaped in that initially g(p) is strictly convex and then is strictly concave (where ∂ 2 g/∂p2 is defined). This can be seen by writing the binomial CDF with parameters z and r as Z 1−p r F (z, r, p) = (r − z) tr−z−1 (1 − t)z , z 0 from which it follows that ∂ 2F r = −(z − (r − 1)p)(r − z) (1 − p)r−z−2 pz 2 ∂p z 39
so ∂ 2 F/∂p2 < 0 for p < π ≡ z/(r − 1) and ∂ 2 F/∂p2 > 0 for p > π. These cut points fall in z, so the segments defined by f (p) as become less convex as p increases and f (p) falls. Because g(0) = 0 and g(1) = 1, and g(·) is s-shaped, there is exactly one inteior point p¯ such that lim g(p) ≤ p¯ ≤ lim+ g(p)
p→¯ p−
p→¯ p
and beyond that point, g(p) > p. If g(f (1/2), r, 1/2) > 1/2 then p¯ < 1/2. If r is even that condition is necessarily satisfied because p∗ < 1/2 and the definition of f (p). If r odd, if af (1/2) =
r+1 , 2
hand f (1/2) ≤
then g(p) is symmetric (g(p) = 1 − g(1 − p)), and p¯ = 1/2.14 If on the other r−1 , 2
then p¯ will be less than 1/2. That occurs when η
r−1 r
1 + (1 − η) ≥ p∗ 2
or equivalently η ≤ r(1 − 2p∗ ). Now for the main proof, consider at+1 ∈ [0, np∗ ]. For z ∈ {0, . . . , r}, let nzt be the number of players with afi,t = z and azt+1 be the number of players with afi,t = z and si,t+1 = A; then azt+1 ∼ B(nzt , πzt ) where πzt = Pr(si,t+1 = A | fi = z, pt ). Finally at+1 = a0t+1 + · · · + art+1 so
E[at+1 ] =
r X
πzt nzt
z=0
Var(at+1 ) =
r X z=0
Var(azt ) =
r X
nzt πzt (1 − πzt ) < n
z=0
r X z=0
14
! πzt (1 − πzt )
<
nr 4
This follows from the symmetry of the binomial probability mass function m(z, n, p), for which m(z, n, p) = m(r − z, n, 1 − p), so m(z, n, 1/2) = m(r − z, n, 1/2), combined with the additional symmetry of af (1/2) = r − af (1/2). For lower η, the second symmetry does not occur, and for even r, the first symmetry does not occur.
40
Now let pt+1 = at+1 /n. E[pt+1 | pt ] is
E[pt+1
f a (pt )−1 r r X X X 1 1 | pt ] = πzt nzt = nzt + (1 − ) nzt n z=0 n z=0 z=f (pt )
Pf (pt )−1 From the strong law of large numbers lim ( z=0 nzt )/n = Pr(fi < f (pt )), so n→∞
lim E[pt+1 | pt ] = (1 − )g(pt ) + (1 − g(pt )) = + (1 − 2)g(pt ).
n→∞
Let ∆(p) ≡ E[pt+1 | pt = p] − p = + (1 − 2)g(p) − p and δ = min{∆(p) : p ≤ p∗ }. Then from Lemma A3 there is some such that δ > 0: consider = p¯ + δ where 0 < δ < 1/2 − p¯. Then for p ≤ p¯, because p¯ < 1/2,
∆(p) = p¯ + δ + (1 − 2¯ p − 2δ)g(p) − p > p¯ + δ − p > δ.
For δ > 0, if p0 = 0, p1 ≥ δ, and in general pt ≥ tδ. Then for t such that pt > p∗ , the system is in D(ωn ) almost surely as n → ∞. Hence WA (n, ) ≤ WA () ≡ p∗ /δ + 1.
A.5
Proof of Proposition 5
Let ωi now be the limit state in which all players play si ∈ S ≡ {s1 , . . . , sm }. I first derive several results on how the radius of limit sets and cost of transitions between limit sets for an unbiased (η = 0) population are related to the same measures for a biased population. For these measures I subscript them with the level of bias, so Rη (ω1 ) is the cost of exiting ω1 for a population with degree of bias η. In stating these lemmas I ignore integer problems, and instead express the statistics as fractions of the population. Two new analytic difficulties arise with m > 2-strategy games: even if all players play s1 or s2 , for example, one needs to 41
consider possibilities in which focal strategies are s3 ; and the fastest transition from ω1 to ω2 may involve some players playing s3 . Some new notation is necessary: aij is the utility received by a player playing si against an opponent playing sj . u(si , σ) is the utility of playing si against a mixed strategy vector σ. ei is the elementary vector assigning 1 to the ith element and 0 to all others. z denotes a state of the population. Let z 0 0 BR(z) = z : if zi > 0, ∃f such that si ∈ arg max u s, ηef + (1 − η) n s now be the set of states in which all players have best-responded to z and some focal observation, and let BRn (z) be the n-fold composition of BR. The possibility set of ω, P (ω) is the states from which there is a strictly positive probability of reaching ω under the unperturbed dynamic: P (ω) = {a ∈ A | Pr(∃T s.t. ∀t > T, at = ω | ao = a) > 0}. It is easy to see that Lemma A4. For a a pure-strategy limit set ω, i) its possibility set is P (ω) = {z : ∃t such that ω ∈ BRt (z)}, and ii) its basin of attraction is D(ωi ) = {z : z ∈ P (ω) and ∀ω 0 6= ω, z ∈ / P (ω 0 )}. The radius of ω is the still the minimum cost of moving out of D(ω), but the coradius and modified coradius of ω are the maximium cost of moving into P (ω) from anywhere in the state space. Because P (ω) is determined by best-response regions, this lemma implies that these costs are the costs of jumps to just beyond the point at which players are indifferent between two strategies.15 15 If only a fraction of the population updated strategies each period, it could be the case that for two adjacent states, BR(z0 ) = BR(z00 ) = ei but z0 and z00 would be in different basins of attraction, for instance
42
Call a path (z0 , z1 , . . . , zn ) a boundary path if it is on the boundary of the state space, i.e., all states in the path are convex combinations of the same two vertices ei and ej of the state space. An interior path is a non-boundary path. Lemma A5. If the least expensive path from ωi to ωj is a boundary path, then
cη (ωi , ωj ) ≤
c0 (ωi , ωj ) − η . 1−η
Proof. The path is along a boundary from ωi to some vertex ek . For η = 0, the cost is the minimum p such that E[U (sj , σ)] = maxs E[U (s, σ)] where σ = pek + (1 − p)ei . Now, for η > 0, players with focal observation sets consisting entirely of sk strategy choices have beliefs σ ˆ = ((1 − η)p0 ei + (η + (1 − η)(1 − p0 ))ek ). At p0 = (p − η)/(1 − η), σ = σ ˆ , so E[u(sj , σ ˆ )] = maxs E[U (s, σ ˆ )], and the state n(p0 ek + (1 − p0 )ei ) ∈ P (ωj ). Lemma A6. If the least expensive path from ωi to ωj is an interior path, then
cη (ωi , ωj ) ≥
c0 (ωi , ωj ) − η . 1−η
Proof. Let σ be the least expensive point in P (ωj ) to reach from ωi at η = 0; let M be the carrier of σ (because σ is the end-point of an interior path, |M | > 2); and let k ∗ = arg maxk∈M \{i} {σk } be the largest component of σ other than σi . P For η = 0, the cost c0 (ωi , ωj ) = σl . Define σ 0 to be the vector σl0 = σ/(1 − η) for l∈M \{i}
l 6= k ∗ and σk0 ∗ = (σk∗ − η)/(1 − η). Let σ ˆ be the belief of players at state σ 0 whose focal observation sets consist entirely of sk strategy choices. By construction, σ ˆ = σ. The cost of if movement towards ei from z0 sometimes reached states z in which BR(z) 6= ei , but the same movement from z00 always stayed in the original best response region.
43
reaching σ 0 , and hence of the transition, is
max{σl0 , 0}, so
P l∈M \{i}
cη (ωi , ωj ) =
P σl −η l∈M \{i}
if η < σk∗
σl l∈M \{i,k∗ }
if η ≥ σk∗
1−η P 1−η
so cη (ωi , ωj ) ≥
R0 (ωi )−η . 1−η
For boundary paths, the cost to move from ωi to ωj can be less than
c0 (ωi ,ωj )−η 1−η
if there is a
focal strategy sf that does relatively better against si than sk itself does (aif −akf < aik −akk ), but this cannot happen with interior paths because such an sf would have been in the support of the mixed strategy. Lemma A7. For any limit point ωi , Rη (ωi , η) ≥
R0 (ωi )−η . 1−η
Proof. I show that the bound is exact if the least expensive path out of D(ω) is a boundary path. For interior paths, from Lemma A6, the bound is satisfied, but not tight for all η. Let p(i, j, k, f ) be the cost of moving from ωi to z ∈ P (ωk ) along the i-j boundary when players’ focal observation sets contain entirely sf observation so that players have beliefs σ ˆ = (ηef (1 − η)(1 − p)ei + (1 − η)pej ) at z. Then p(i, j, k, f ) solves E[u(si ; σ ˆ )] = E[u(sk ; σ ˆ )], or
ηaif + (1 − η){(1 − p)aii + paij } = ηakf + (1 − η){(1 − p)aki + pkj }
so p(i, j, k, f ) =
aii − aki aii − aki + akj − aij
+
44
η 1−η
aif − akf aii − aki + akj − aij
.
(4)
Provided that the least expensive exit from ωi is a boundary transition, R(ωi ) is proportional to the smallest p(i, j, k, f ) available: R(ωi ) = min p(i, j, k, f ) = min j,k,f ∈S
k,f ∈S
Let j ∗ solve the minimization in parenthes.
min p(i, j, k, f ) . j∈S
(5)
Because the j term only appears in the
denominator in the expression (4), the optimal j ∗ is a function of k alone. Fixing k, j ∗ = arg max{akj − aij } and f ∗ = arg min{aif − akf } = arg max{akf − aif }, the same optimization problem, so for given k, j ∗ = f ∗ . Let j ∗ and k ∗ solve (5) for η = 0. Then the same j ∗ , k ∗ , and f ∗ = j ∗ solve (5) for η > 0. Hence η R0 (ωi ) − η (R0 (ωi ) − 1) = . 1−η 1−η
Rη (ωi ) = p(i, j ∗ , k ∗ , j ∗ ) = R0 (ωi ) +
Now I construct an upper bound for CR∗η (ω) measures. Lemma A8. If (ωz0 , ωz1 , ωzT ) is a path made up of boundary paths from from ωz0 to ωz1 , ωz1 to ωz2 , and so on, ending with ωzT , and it is the most expensive path to ωzT (c∗η (ωz0 , ωzT ) ≥ n ∗ o c (ω ,ω )−η c∗η (ω 0 , ωzT ) for all ω 0 ∈ Ω), then c∗η (ω0 , ωT ) ≤ max 0 z01−ηzT , 0 . Proof. The proof is by induction on the property
∗
c (ωz0 , ωzt ) ≤ max
c∗0 (ωz0 , ωzt ) − η ,0 1−η
(P)
Because a direct path c∗ (ωz0 , ωz1 ) = c(ωz0 , ωz1 ), Lemma A5 implies (P) holds for t = 1. There are two useful consequences of the assumption that the path for ωz0 to ωzT is the most expensive path to ωzT . It must at least as expensive as a path starting at ωzt , i.e., 45
c∗η (ωz0 , ωzT ) ≥ c∗η (ωzt , ωzT ) and so c∗η (ωz0 , ωzT ) = c∗η (ωz0 , ωzt ) − Rη (ωzt ) + c∗η (ωzt , ωzT ) ≥ c∗η (ωzt , ωzT ) c∗η (ωz0 , ωzt ) ≥ Rη (ωzt ) c∗0 (ωz0 , ωzt ) − η R0 (ωzt ) − η ≥ c∗η (ωz0 , ωzt ) ≥ 1−η 1−η so R0 (ωzt ) ≤ c∗0 (ωz0 , ωzt ) if the path from ωz0 is more expensive. Likewise, because c∗η (ωz0 , ωzT ) ≥ c∗η (ωzt , ωzT ), c∗η (ωz0 , ωz(t+1) ) − Rη (ωz(t+1) ) + c∗η (ωz(t+1) , ωzT ) ≥ cη (ωzt , ωz(t+1) ) + c∗η (ωz(t+1) , ωzT ) − Rη (ωz(t+1) ) cη (ωzt , ωz(t+1) ) ≤ c∗η (ωz0 , ωz(t+1) ) c0 (ωzt , ωz(t+1) ) − η c∗ (ωz0 , ωz(t+1) ) − η ≤ c∗η (ωz0 , ωz(t+1) ) ≤ 0 1−η 1−η so c0 (ωzt , ωz(t+1) ) ≤ c∗0 (ωz0 , ωz(t+1) ). Now assume P holds for t. By definition, c∗η (ωz0 , ωz(t+1) ) = c∗η (ωz0 , ωzt ) + cη (ωzt , ωz(t+1) ) − R(ωzt ). Let η¯ = max{c0 (ωz0 , ωz1 ), . . . , c0 (ωz(t−1) , ωzt ), R0 (ωz1 ), . . . , R(ωz(t−1) )} be the largest η for which c∗η (ωz0 , ωzt ) > 0. Consider the case in which η¯ ≤ R0 (ωzt ) ≤ c0 (ωzt , ωz(t+1) ). The other cases are similar and I omit them.16
c∗η (ωz0 , ωz(t+1) ) ≤
c (ω ,ω )−R0 (ωzt ) c∗0 (ωz0 ,ωzt )−η + 0 zt z(t+1) 1−η 1−η c0 (ωzt ,ωz(t+1) )−R0 (ωzt ) 1−η
c0 (ωzt ,ωz(t+1) )−η 1−η 0
if η < η¯ if η¯ < η ≤ R0 (ωzt ) if R0 (ωzt ) < η ≤ c0 (ωzt , ωz(t+1) ) if η > c0 (ωzt , ωz(t+1) )
16
Note that R0 (ωzt ) is necessarily no larger than c0 (ωzt , ωz(t+1) ) but η¯ can have any relationship to the other quantities if t is large enough.
46
For η < η¯,
c∗η (ωz0 , ωz(t+1) )
c∗0 (ωz0 , ωzt ) − η + c0 (ωzt , ωz(t+1) ) − R0 (ωzt ) c∗0 (ωz0 , ωz(t+1) ) − η ≤ = . 1−η 1−η
For η¯ < η ≤ R0 (ωzt ), c0 (ωzt , ωz(t+1) ) − R0 (ωzt ) 1−η c∗0 (ωz0 , ωz(t+1) ) − c∗0 (ωz0 , ωzt ) = 1−η c∗0 (ωz0 , ωz(t+1) ) − R0 (ωzt ) c∗ (ωz0 , ωz(t+1) ) − η ≤ ≤ 0 . 1−η 1−η
c∗η (ωz0 , ωz(t+1) ) ≤
Finally, in the range R0 (ωzt ) < η ≤ c(ωzt , ωz(t+1) ), c∗η (ωz0 , ωz(t+1) ) ≤
c∗ (ωz0 , ωz(t+1) ) − η c0 (ωzt , ωz(t+1) ) − η ≤ 0 1−η 1−η
because the path from ωz0 is more costly than the path from ωzt . Therefore property (P) holds for t + 1 as well. Lemma A7 corresponds to part (i) of Proposition 5 while Lemma A8 contains the main insight for part (ii) of the proposition. To complete the argument for part (ii), consider the possibly indirect transition path to ω from ω 0 which determines CR0 (ω). Lemma A6 implies that if this path is a boundary path then for η > 0, this transition path is less expensive than any interior path, so CR∗η (ω) = maxω0 ∈Ω c∗η (ω 0 , ω) ≤
47
c∗0 (ω)−η 1−η
from Lemma A8.
B
Wait Time Calculations
I calculate WA (n, ) using standard first-step analysis techniques. Let n0 = bp∗ nc denote the lowest state in D(ωn ) and let P be an n0 × n0 be a matrix with entries
pij =
Pr(at+1 = j | at = i) if i, j < n0 Pr(at+1 ≥ j | at = i) if i < n0 and j = n0 0
0 1
.
0
if i = n and j < n if i = j = n0
P is a Markov transition matrix formed by starting with the transition matrix of the perturbed evolutionary process and then treating all states a ∈ D(ωn ) as a single absorbing state. Let Q be the matrix of transition probabilities between transient states of P , obtained by retaining the first n0 − 1 rows and columns of P . Then let T be the (n0 − 1) × (n0 − 1) matrix with elements tij equal to the expected number of times the system is in state j, given that it starts in state i. T = I + QT , and hence T = (I − Q)−1 . Then the expected wait time Pn0 −1 WA (n, ) = j=0 t0,j .
48
