A cognitive principle of least effort explains many cognitive biases Nisheeth Srivastava & Paul R Schrater University of Minnesota March 31, 2011 Abstract We formulate a model of decision-making based on alternative rationality criteria that retains the mathematical tractability of classical rational choice models. Although rational, our model generatively replicates several heuristics shown to be effective in explaining decision-making behaviors hitherto considered irrational. In particular, simulations of our model replicate three classic experimental studies spanning distinctive families of cognitive biases, viz. probabilistic sub-additivity leading to a fourfold pattern of risk aversion, confirmatory positive hypothesis selection and serial ordering effects.

Motivation: Expected utility, the canonical framework for decision theory in the past century, has been shown to be inaccurate and unrealistic by multiple theoretical and empirical studies in the past three decades. However, replacement efforts have not so far met with success. On the one hand, experimental economists have demonstrated the existence of multiple behaviors unexplained by expected utility formulations of rational choice and neuroeconomists have demonstrated the insufficiency of traditional hedonic models of reward in describing the reality of subjects’ motivations in choice selection tasks. On the other, formal models of decision-making are restricted by concerns over mathematical tractability from making significant adjustments to the classical account. This gap between observation and theory has resulted in an unsatisfactory compromise of using ad hoc evolutionarily-motivated heuristics in different decision domains. The need for a coherent mathematical model of decision-making, both mathematically tractable and in concordance with the neuro-biology of choice selection processes, has never been greater. Research question: We examine computationally whether a mathematically tractable decision theory built on a principle of least cognitive effort can present a more plausible and principled explanation for human choice behavior. In particular, we conjecture that the cognitive mechanisms underlying decisionmaking, under evolutionary selection pressure, are likely to have been optimized to satisfy some internal criterion for predictive efficiency. We explore whether information-theoretic measures of predictive ability can capture this notion of optimality, and whether such a conjecture can lead to a good model of decisionmaking. Methodology: Using evolutionary arguments, we propose that it is more realistic to assume that humans solve a dual problem: they attempt to minimize cognitive processing costs while holding rewardgathering performance above some satisfactory level. Further, we suggest that it is more realistic to imbue biological agents with the ability to infer the relative quality of possible outcomes, an epistemic device recently described in the literature as choice queries, rather than assume the existence of absolute reward signals associated with environmental states. We model cognitive processing costs in a sequential decisionmaking setting as memory access costs. We quantify memory access costs using a novel information-theoretic encoding criterion, motivated by the MDL statistical model selection principle [1]. The intuition behind this structure stems from a random access memory model, wherein objects in memory that occupy less space are harder to access and, from coding theory principles, objects that are exceptional in a predictive sense require more space to store. We show that it is possible to optimize our model’s objective function in a way that 1

strongly resembles on-policy model-free reinforcement learning, but with striking statistical dissimilarities from existing RL algorithms. Results: Simulations of populations of agents behaving according to our model replicate empirical results from multiple families of cognitive biases with no prior common explanation. The replication of disparate cognitive phenomena connected with the decisions of human subjects suggests that our alternative rationality criterion captures hitherto unknown deeper structures in the human decision-making process. As a corollary, we show that several putatively irrational cognitive ‘biases’ are rational when viewed from our novel cognitively motivated definition of rationality1 . Furthermore, we demonstrate that agents using our algorithm for choice selection generatively learn interesting strategies in two-player games, including a mean tit-for-tat strategy in prisoners’ dilemma settings, and replicate important predictions from Fehr-Schmidt social utility theory. Discussion: [3] have shown that using a choice query that merely indicates which of several options is preferable instead of cardinal utilities retains much of the functionality of the utility maximization paradigm. As such, it is very exciting to see that a principled decision model can be built using choice queries. The fact that our model makes predictions contrary to classical theory, but validated by experimental data on human subjects provides support for the view that choice queries are a better representation of inferring value than cardinal utilities. Since our choice model makes no axiomatic assumptions about the nature of utilities, comparing its neuro-biological plausibility against existing models emerging from the reinforcement learning [4] and neuroeconomics [5] literature becomes an important research question. Our results are of interest to the neuroscience community in that they make testable predictions about the nature of belief encoding in human long-term memory. For example, the information-theoretic optimality postulated in our model appears to be borne out by emerging empirical results from [6]. Our results should also be of interest to behavioral economists and neuroeconomists, since they suggest a common and evolutionarily principled etiology for behaviors that have commonly been interpreted as irrational and biased. Finally, we observe the formal homology of our model with the classical decision theory framework increases its accessibility as a hypothesis in social simulations.

References [1] P. Grnwald, The Minimum Description Length Principle. MIT Press, June 2007 [2] N. Srivastava and P. R. Schrater (2011). A predictive model for self-motivated decision-making behavior. In Proceedings of BRIMS 2011. [3] P. Viappiani and C. Boutilier (2010). Optimal Bayesian Recommendation Sets and Myopically Optimal Choice Query Sets. Advances in Neural Information Processing Systems 23 [4] N. Daw, J. P. O’Doherty, P. Dayan, B. Seymour, and R. J.Dolan, Polar Exploration: Cortical Substrates for Exploratory Decisions in Humans, Nature, 441 (2006), 876879 [5] A. Caplin and M. Dean (2008). Axiomatic Methods, Dopamine and Reward Prediction Error. Current Opinion in Neurobiology, 18(2): 197-202 [6] J. Rubin, I. Nelken, N. Tishby. Cortical representation of predictive information. Cosyne 2011

1A

more complete description of some of these results was recently presented in [2]

2