Inbox: - https://forum.effectivealtruism.org/posts/kCBQHWqbk4Nrns8P7/model-based-policy-analysis-under-deep-uncertainty [My mildly scathing review of their book](https://sun.pjh.is/tetlock-vs-king-and-kay-on-probabilistic-reasoning). ## Radical Uncertainty book blurb Much economic advice is bogus quantification, warn two leading experts in this essential book, now with a preface on COVID-19. **Invented numbers offer a false sense of security**; we need instead robust narratives that give us the confidence to manage uncertainty. Some uncertainties are resolvable. The insurance industry’s actuarial tables and the gambler’s roulette wheel both yield to the tools of probability theory. **Most situations in life, however, involve a deeper kind of uncertainty, a radical uncertainty for which historical data provide no useful guidance to future outcomes. Radical uncertainty concerns events whose determinants are insufficiently understood for probabilities to be known or forecasting possible.** The limits of certainty demonstrate the power of human judgment over artificial intelligence. **In most critical decisions there can be no forecasts or probability distributions on which we might sensibly rely.** Instead of inventing numbers to fill the gaps in our knowledge, we should adopt business, political, and personal strategies that will be robust to alternative futures and resilient to unpredictable events. Within the security of such a robust and resilient reference narrative, uncertainty can be embraced, because it is the source of creativity, excitement, and profit. ## Swagman short interview You can't attach a probability to inventing the wheel because if you can you've already invented it. People don't naturally think probabilistically. In very complicated situations the only thing you can do is create narratives. Things we know roughly but not precisely. More likely than not means ">50%". Holden is saying >10% on PASTA within 15 years. Allison: 12 / 16 cases Where rising power challenged established power, result was war. On this basis, he says that war between US and China is more likely than not. Kay: that's a good way of framing it. Ppl should not say 12/16. Likelyhood: ordinal variable Probability: cardinal variable It's okay to say something is more likely or less likely, or that this information I've just received makes it more likely or less likely. Advocate of "vague verbiage". On the margin we have too much false precision, and that's a worse problem than people misunderstanding each other. Critique of Tetlock: frame questions precisely enough that there are precise answers. But that's not really the question we want answered. We want to ask broader questions. Framing questions more as puzzles rather than mysteries. Puzzle = well formulate question that has a Message: "Manage risk, embrace uncertainty." # Radical Uncertainty: Decision-making for an Unknowable Future *Highlight [page 116]:* One of the greatest reasons why so few people understand themselves is that most writers are always teaching men what they should be, and hardly ever trouble their heads with telling them what they really are. —BERNARD MANDEVILLE , Fable of the Bees 1 **The overriding objective was to ensure that the reference narrative was robust and resilient. The key to managing risk is the identification of reference narratives which have these properties of robustness and resilience.** We believe the best way to understand attitudes to risk is through the concept of a **reference narrative**,**a story which is an expression of our realistic expectations.** **There is an implied measure of risk in such assessment – an outcome can fall short of expectations by a narrow margin or a wide one. The scale of that risk may or may not be quantifiable, before or after the event.** **Insurance is based not on calculations of expected value, but on the desire to protect the reference narrative of the insured.** We cannot act in anticipation of every unlikely possibility. So we select the unlikely events to monitor. **Not by some metarational calculation** which considers all these remote possibilities and calculates their relative importance, **but by using our judgement and experience**. ## Selected highlights The distinction between a **small world** in which people can solve problems by maximising expected utility and the **large world** in which people actually live is crucial Consistency of choices was critical among these axioms. Consistent behaviour requires that if you accept (or reject) a gamble once, you should accept (or reject) the same gamble every time it is offered The philosopher **Anthony Appiah coined the term ‘cognitive angels’ to describe idealised agents who can maximise expected utility correctly, and contrasted them with real people who take time to compute and make errors in the process**. Richard Thaler, winner in 2017 of the Nobel Prize in Economics, similarly distinguished ‘econs’ and ‘humans’. 18 We do not regard the pursuit of expected utility maximisation – in a world of radical uncertainty – as characteristic of an angel. No one has ever enjoyed so much access to data, and so much expertise to tell him what he might need to know, as is available to the President of the United States. But even with such resources, an exercise of this kind is impossible. Obama was not optimising. He was not maximising his subjective expected utility, or that of the nation. He could not conceivably have held the information which would have enabled him to do so. How could he, in the face of so many ill-defined uncertainties? Steve -obs was not watching a Bayesian dial: he was waiting until he recognised ‘the next big thing’. And Winston Churchill also played a waiting game as he saw the United States gradually dragged into war – and did his utmost to accelerate American entry. He sat and listened to conflicting accounts and evidence until he felt he had enough information – knowing that he could expect only limited and imperfect information – to make a decision. That is how good decisions are made in a world of radical uncertainty, as decision-makers wrestle with the question ‘What is going on here?’ We do not know whether Obama walked into the fateful meeting with a prior probability in his mind: we hope not He sat and listened to conflicting accounts and evidence until he felt he had enough information – knowing that he could expect only limited and imperfect information – to make a decision. expectations which take a narrative rather than probabilistic form. We believe the best way to understand attitudes to risk is through the concept of a **reference narrative**,**a story which is an expression of our realistic expectations.** A reference narrative governed the meeting at which President Obama ordered the SEALs to Abbottabad. The helicopters would land in the compound, the men would fight their way into the building. We suppose, but do not know, that the unspoken premise was that bin Laden would be killed in the attack. Dead or alive, he would be flown out of Pakistan in US custody. That reference narrative describes, more or less, what actually happened. But many things might have derailed that narrative. The operation might have suffered the equipment and logistical failures which had grounded the 1979 mission to rescue the Tehran hostages. Bin Laden might not have been in the compound, because US intelligence was faulty or because he was absent at the time of the raid. The President and his advisers debated these risks and discussed appropriate responses. The most difficult issue was how to handle a situation where the operation was quickly detected by the Pakistani military authorities and confronted with an armed response. **The overriding objective was to ensure that the reference narrative was robust and resilient. The key to managing risk is the identification of reference narratives which have these properties of robustness and resilience.** Risk is failure of a projected narrative, derived from realistic expectations, to unfold as envisaged. The happy father anticipating his daughter’s wedding has in mind a reference narrative in which events go ahead as planned. He recognises a variety of risks – the prospective bridegroom has cold feet, a torrential downpour drenches the guests. **There is an implied measure of risk in such assessment – an outcome can fall short of expectations by a narrow margin or a wide one. The scale of that risk may or may not be quantifiable, before or after the event.** **Insurance is based not on calculations of expected value, but on the desire to protect the reference narrative of the insured.** The reference narrative is one in which we arrive on holiday with all our luggage, the mobile phone is in our pocket, the washing machine is working and the College silver is available for feasts. *Highlight [page 110]:* insurance often protects the reference narrative of employees rather than the organisation they work for. The members of the College Finance Committee knew they would be blamed if uninsured silver was stolen, but not thanked for saving the cost of the premium. *Highlight [page 111]:* The primary purpose of risk management is often to protect the reference narrative of individuals within the organisation rather than the organisation itself. *Highlight [page 112]:* **We cannot act in anticipation of every unlikely possibility. So we select the unlikely events to monitor. Not by some metarational calculation which considers all these remote possibilities and calculates their relative importance, but by using our judgement and experience.** *Highlight [page 116]:* One of the greatest reasons why so few people understand themselves is that most writers are always teaching men what they should be, and hardly ever trouble their heads with telling them what they really are. —BERNARD MANDEVILLE , Fable of the Bees 1 ## All highlights *Highlight [page 97]:* The choices we make between uncertain events are typically far more complex than those observed in elementary games of chance. The Viniar problem – the mistake of believing you have more knowledge than you do about the real world from the application of conclusions from artificial models – runs deep. *Underline [page 97]:* The Viniar problem – the mistake of believing you have more knowledge than you do about the real world from the application of conclusions from artificial models – runs deep. *Highlight [page 98]:* Nineteenth-century economics was developed in the context of the utilitarianism of the English philosophers -eremy Bentham and -ohn Stuart Mill. Individuals sought to maximise their utility and moral actions served to maximise the sum of such utilities – ‘the greatest happiness of the greatest number’. The Oxford economist F. Y. Edgeworth, a pioneer of the use of mathematical reasoning in economics in the late nineteenth century, visualised the ‘hedonimeter’ which would measure pleasure and pain as objectively as a thermometer would measure temperature. These writers went too far, and no one today believes in the possibility of hedonimeters, or that shoppers peruse the aisles using their smartphones to make calculations which maximise their utility. *Underline [page 98]:* The Oxford economist F. Y. Edgeworth, a pioneer of the use of mathematical reasoning in economics in the late nineteenth century, visualised the ‘hedonimeter’ which would measure pleasure and pain as objectively as a thermometer would measure temperature. *Highlight [page 98]:* **Consistency of choices was critical among these axioms.** This approach proved fruitful in the understanding of many real-world problems. It allowed economists to analyse many questions concerning how changes in prices, reflecting changes in supply conditions or economic policies, would affect the allocation of resources in a market economy. And economics textbooks are full of examples of the successful application of such techniques. Several leaps were required to extend this thinking from consumer choice between goods to decision-making under uncertainty. Samuelson at first resisted, but then applauded, such extension. And that change of heart was how he was able to criticise his reluctant-to-wager colleague for his inconsistency. **Consistent behaviour requires that if you accept (or reject) a gamble once, you should accept (or reject) the same gamble every time it is offered**. Just as consistent behaviour requires that if you prefer today to travel to work by car rather than train, you will choose the car today, and tomorrow, and the day after. *Underline [page 98]:* Consistency of choices was critical among these axioms. *Underline [page 98]:* Consistent behaviour requires that if you accept (or reject) a gamble once, you should accept (or reject) the same gamble every time it is offered. *Highlight [page 99]:* In their classic work The Theory of Games and Economic Behavior , von Neumann and his Princeton colleague Oskar Morgenstern sought to establish that probabilistic reasoning could provide a coherent and rigorous framework for rational decision-making in a world of uncertainty. *Highlight [page 100]:* **The distinction between a small world in which people can solve problems by maximising expected utility and the large world in which people actually live is crucial**, and we shall refer to it frequently in later chapters. Savage went on to explain that he believed his approach might be used ‘to attack relatively simple problems of decision by artificially confining attention to so small a world that the “look before you leap” principle can be applied there’. *Underline [page 100]:* The distinction between a small world in which people can solve problems by maximising expected utility and the large world in which people actually live is crucial, and we shall refer to it frequently in later chapters. *Underline [page 100]:* the “look before you leap” principle *Highlight [page 100]:* Savage’s caution over the scope of his analysis was not shared by economists who have since been happy not only to adopt the assumption that in a world of uncertainty individuals optimise by maximising their expected utility, but to claim that the resulting models have direct application to policies appropriate for large worlds *Underline [page 100]:* Savage’s caution over the scope of his analysis was not shared by economists who have since been happy not only to adopt the assumption that in a world of uncertainty individuals optimise by maximising their expected utility, but to claim that the resulting models have direct application to policies appropriate for large worlds. *Highlight [page 101]:* When Milton Friedman retired from Chicago in 1976, Gary Becker acquired the mantle of academic leader of the Chicago School. And Becker’s aspirations for the application of his ideas were as ambitious as Friedman’s. ‘All human behavior’, he wrote, ‘can be regarded as involving participants who maximize their utility from a stable set of preferences and accumulate an optimal amount of information and other inputs in a variety of markets. If this argument is correct, the economic approach provides a unified framework for understanding behavior that has long been sought by and eluded Bentham, Comte, Marx, and others.’ *Highlight [page 102]:* Bernoulli’s resolution of the St Petersburg paradox was that a gamble is worthwhile if and only if it maximises expected utility rather than expected wealth. And so decision-making using probabilistic reasoning came to be equated with maximisation of expected utility. Pascal, with his wager on the existence of God, had foreshadowed the idea of deciding on a gamble by multiplying the gain or loss, measured in terms of happiness rather than money, by the chance of that gain or loss occurring. *Underline [page 102]:* deciding on a gamble by multiplying the gain or loss, measured in terms of happiness rather than money *Highlight [page 104]:* The philosopher **Anthony Appiah coined the term ‘cognitive angels’ to describe idealised agents who can maximise expected utility correctly, and contrasted them with real people who take time to compute and make errors in the process**. Richard Thaler, winner in 2017 of the Nobel Prize in Economics, similarly distinguished ‘econs’ and ‘humans’. 18 We do not regard the pursuit of expected utility maximisation – in a world of radical uncertainty – as characteristic of an angel. *Underline [page 104]:* Anthony Appiah coined the term ‘cognitive angels’ to describe idealised agents who can maximise expected utility correctly, *Highlight [page 104]:* They learnt about Bayesian reasoning in the classroom but did not apply it in their lives because it was rarely useful – in planning their lives and their careers they simply did not have the information needed to compute their expected utilities. *Underline [page 104]:* it in their lives because it was rarely useful – in planning their lives and their careers they simply did not have the information needed to compute their expected utilities. *Highlight [page 105]:* No one has ever enjoyed so much access to data, and so much expertise to tell him what he might need to know, as is available to the President of the United States. But even with such resources, an exercise of this kind is impossible. Obama was not optimising. He was not maximising his subjective expected utility, or that of the nation. He could not conceivably have held the information which would have enabled him to do so. How could he, in the face of so many ill-defined uncertainties? Steve -obs was not watching a Bayesian dial: he was waiting until he recognised ‘the next big thing’. And Winston Churchill also played a waiting game as he saw the United States gradually dragged into war – and did his utmost to accelerate American entry. *Underline [page 105]:* He was not maximising his subjective expected utility, or that of the nation. He could not conceivably have held the information which would have enabled him to do so. How could he, in the face of so many ill-defined uncertainties? *Underline [page 105]:* Steve -obs was not watching a Bayesian dial: he was waiting until he recognised ‘the next big thing’. *Highlight [page 105]:* We do not know whether Obama walked into the fateful meeting with a prior probability in his mind: we hope not. He sat and listened to conflicting accounts and evidence until he felt he had enough information – knowing that he could expect only limited and imperfect information – to make a decision. That is how good decisions are made in a world of radical uncertainty, as decision-makers wrestle with the question ‘What is going on here?’ *Underline [page 105]:* We do not know whether Obama walked into the fateful meeting with a prior probability in his mind: we hope not *Underline [page 105]:* He sat and listened to conflicting accounts and evidence until he felt he had enough information – knowing that he could expect only limited and imperfect information – to make a decision. *Highlight [page 107]:* expectations which take a narrative rather than probabilistic form. *Underline [page 107]:* expectations which take a narrative rather than probabilistic form. *Highlight [page 107]:* We believe the best way to understand attitudes to risk is through the concept of a reference narrative , a story which is an expression of our realistic expectations. *Underline [page 107]:* reference narrative , a story which is an expression of our realistic expectations. *Highlight [page 108]:* A reference narrative governed the meeting at which President Obama ordered the SEALs to Abbottabad. The helicopters would land in the compound, the men would fight their way into the building. We suppose, but do not know, that the unspoken premise was that bin Laden would be killed in the attack. Dead or alive, he would be flown out of Pakistan in US custody. That reference narrative describes, more or less, what actually happened. But many things might have derailed that narrative. The operation might have suffered the equipment and logistical failures which had grounded the 1979 mission to rescue the Tehran hostages. Bin Laden might not have been in the compound, because US intelligence was faulty or because he was absent at the time of the raid. The President and his advisers debated these risks and discussed appropriate responses. The most difficult issue was how to handle a situation where the operation was quickly detected by the Pakistani military authorities and confronted with an armed response. The overriding objective was to ensure that the reference narrative was robust and resilient. The key to managing risk is the identification of reference narratives which have these properties of robustness and resilience. *Underline [page 108]:* The overriding objective was to ensure that the reference narrative was robust and resilient. The key to managing risk is the identification of reference narratives which have these properties of robustness and resilience. *Highlight [page 108]:* Risk is failure of a projected narrative, derived from realistic expectations, to unfold as envisaged. The happy father anticipating his daughter’s wedding has in mind a reference narrative in which events go ahead as planned. He recognises a variety of risks – the prospective bridegroom has cold feet, a torrential downpour drenches the guests. There is an implied measure of risk in such assessment – an outcome can fall short of expectations by a narrow margin or a wide one. The scale of that risk may or may not be quantifiable, before or after the event. *Highlight [page 110]:* Insurance is based not on calculations of expected value, but on the desire to protect the reference narrative of the insured. The reference narrative is one in which we arrive on holiday with all our luggage, the mobile phone is in our pocket, the washing machine is working and the College silver is available for feasts. *Underline [page 110]:* Insurance is based not on calculations of expected value, but on the desire to protect the reference narrative of the insured. *Highlight [page 110]:* insurance often protects the reference narrative of employees rather than the organisation they work for. The members of the College Finance Committee knew they would be blamed if uninsured silver was stolen, but not thanked for saving the cost of the premium. *Highlight [page 111]:* The primary purpose of risk management is often to protect the reference narrative of individuals within the organisation rather than the organisation itself. *Underline [page 111]:* The primary purpose of risk management is often to protect the reference narrative of individuals within the organisation rather than the organisation itself. *Highlight [page 112]:* We cannot act in anticipation of every unlikely possibility. So we select the unlikely events to monitor. Not by some metarational calculation which considers all these remote possibilities and calculates their relative importance, but by using our judgement and experience. *Highlight [page 116]:* One of the greatest reasons why so few people understand themselves is that most writers are always teaching men what they should be, and hardly ever trouble their heads with telling them what they really are. —BERNARD MANDEVILLE , Fable of the Bees 1 *Underline [page 116]:* One of the greatest reasons why so few people understand themselves is that most writers are always teaching men what they should be, and hardly ever trouble their heads with telling them what they really are. *Highlight [page 116]:* he approach to decision-making under uncertainty developed by von Neumann and Morgenstern and elaborated by Friedman and Savage in the 1940s sets out a definition of ‘rationality’ based not on observation or introspection, but on a set of a priori axioms. This way of thinking we will describe as ‘axiomatic rationality’. It has the logical consequence that there is something which might be described as ‘subjective expected utility’ which individuals who are ‘rational’ are maximising. Obedience to these axioms, it was claimed, defined ‘rational’ behaviour. This is not a particularly obvious way to define ‘rationality’ and it is certainly not the only possible approach. It is, however, one which has come to dominate economics. *Highlight [page 118]:* the approach pioneered by Kahneman and Tversky adopted a markedly different stance. The subject of their critique is the decision-maker, not the model of decision-making. If the world does not conform to the model, the failure is not a failure of the model but a failure of the world, or to be precise, of the people the model is intended to describe. *Highlight [page 118]:* Since the word ‘rationality’ is powerful, it should be used with great care. But what does it mean to act rationally? Ordinary usage suggests two characteristics of rational judgement or action. First, the judgement or action would be based on beliefs about the world which were reasonable. Not necessarily correct beliefs *Highlight [page 119]:* A second requirement of rationality is an element of internal logic or consistency. The judgement or action is appropriate given the beliefs about the world which give rise to it. This proposition requires care in interpretation. It may be difficult to distinguish errors in reasoning from mistakes in belief. *Highlight [page 119]:* The axioms of choice under uncertainty do not enjoy any monopoly on the term ‘rationality’. *Highlight [page 119]:* Rational behaviour is not defined by conformity with a set of axioms set down even by such distinguished thinkers as -ohn von Neumann and Milton Friedman. *Highlight [page 119]:* Charles Sanders Pierce, a founder of the American school of pragmatist philosophy, distinguished three broad styles of reasoning. Deductive reasoning reaches logical conclusions from stated premises. *Highlight [page 120]:* Inductive reasoning is of the form ‘analysis of election results shows that they normally favour incumbent parties in favourable economic circumstances and opposition parties in adverse economic circumstances’. Since economic conditions in the United States in 2016 were neither particularly favourable nor unfavourable, we might reasonably have anticipated a close result. Inductive reasoning seeks to generalise from observations, and may be supported or refuted by subsequent experience. Abductive reasoning seeks to provide the best explanation of a unique event. For example, an abductive approach might assert that Donald Trump won the 2016 presidential election because of concerns in particular swing states over economic conditions and identity, and because his opponent was widely disliked. Deductive, inductive and abductive reasoning each have a role to play in understanding the world, and as we move to larger worlds the role of the inductive and abductive increases relative to the deductive. And when events are essentially one-of-a-kind, which is often the case in the world of radical uncertainty, abductive reasoning is indispensable. Although the term ‘abductive reasoning’ may be unfamiliar, we constantly reason in this way, searching for the best explanation of what we see: ‘I think the bus is late because of congestion in Oxford Street’. But the methods of decision analysis we have described in earlier chapters are derived almost entirely from the deductive reasoning which is relevant only in small worlds. *Underline [page 120]:* the methods of decision analysis we have described in earlier chapters are derived almost entirely from the deductive reasoning which is relevant only in small worlds. *Highlight [page 120]:* We have considerable sympathy with the concept of irrationality put forward by the Israeli economist Itzhak Gilboa: ‘A mode of behavior is irrational for a decision-maker, if, when the latter is exposed to the analysis of her choices, she would have liked to change her decision, or to make different choices in similar future circumstances.’ 8 Someone who was waiting for a bus in the belief that the normal timetable was operating would probably agree that they had been unfortunate or even foolish once it was explained that today was a public holiday with no service. But if they remained at the stop after learning this, they would be irrational. *Highlight [page 121]:* logic derived from reasonably maintained premises can only ever take us so far. Under radical uncertainty, the premises from which we reason will never represent a complete description of the world. There will be different actions which might properly be described as ‘rational’ given any particular set of beliefs about the world. As soon as any element of subjectivity is attached either to the probabilities or to the valuation of the outcomes, problems cease to have any objectively correct solution. *Highlight [page 128]:* Behavioural economics has contributed to our understanding of decision-making in business, finance and government by introducing observation of how people actually behave. But, like the proselytisers for the universal application of probabilistic reasoning, practitioners and admirers of behavioural economics have made claims far more extensive than could be justified by their findings. *Highlight [page 129]:* We believe that it is time to move beyond judgemental taxonomies of ‘biases’ derived from a benchmark which is a normative model of human behaviour deduced from implausible a priori principles. And ask instead how humans do behave in large worlds of which they can only ever have imperfect knowledge. *Highlight [page 129]:* ask instead how humans do behave in large worlds of which they can only ever have imperfect knowledge. *Highlight [page 129]:* a philosophy of nudging carries the risk that nudgers claim to know more about an uncertain world than they and their nudgees do or could know. *Highlight [page 129]:* As *Highlight [page 130]:* we explained in chapter 1 , it is extremely difficult to assess how much any particular individual should invest in a pension plan. *Highlight [page 130]:* danger of well-meaning illiberalism. Tversky was interested in what he called ‘natural stupidity’, and prone to find it in those who disagreed with him. *Highlight [page 130]:* The problem of making good decisions in large worlds is generally not the difficulty of calculating the logical consequences of agreed premises and a well-defined set of alternative actions – a task a computer can now perform better than humans. It is the problem of context – the impossibility of knowing all the feasible choices and the full detail of the environment in which these choices will take effect. *Highlight [page 130]:* The human brain is not a computer implementing an axiomatic decision-making process, and as a result is a better decision-maker in many complex situations. *Highlight [page 130]:* Simon recognised that radical uncertainty prevented people from behaving in the optimising manner defined by a priori axioms. And so he argued that ‘more than minor tampering with existing optimization theory is called for’. He was anticipating, but not preventing, the subsequent development of a large literature based on such minor tampering. *Underline [page 130]:* He suggested that one way in which people might approach decisions in a radically uncertain world was to use a rule of thumb to search for a ‘good enough’ outcome. Such behaviour was described as ‘satisficing’, and in practice can deliver superior outcomes to actions selected by optimising behaviour. The reason is that to pretend to optimise in a world of radical uncertainty, it is necessary to make simplifying assumptions about the real world. *Highlight [page 130]:* He suggested that one way in which people might approach decisions in a radically uncertain world was to use a rule of thumb to search for a ‘good enough’ outcome. Such behaviour was described as ‘satisficing’, and in practice can deliver superior outcomes to actions selected by optimising behaviour. The reason is that to pretend to optimise in a world of radical uncertainty, it is necessary to make simplifying assumptions about the real world. If these assumptions are *Highlight [page 131]:* wrong – as in a world of radical uncertainty they are almost certain to be optimisation yields the wrong results, just as someone searching for their keys under the streetlamp because that is where the light is best makes the error of substituting a well-defined but irrelevant problem for the less well-defined problem he actually faces. *Underline [page 131]:* the error of substituting a well-defined but irrelevant problem for the less well-defined problem he actually faces. *Highlight [page 131]:* Economists have adapted the phrase ‘bounded rationality’ to mean something very different from Simon’s description as the consequence of radical uncertainty. They have instead used it to describe the cost of processing information, which then acts as an additional constraint in an optimisation problem. Bounded rationality, in this sense, adds to the optimisation calculation the costs and benefits of obtaining the information which we choose not to have. Of course, this is not what Simon meant. Nor, indeed, does it make much sense as a description of any process with practical application. The implications of bounded rationality are not represented by adding computational costs to an optimisation problem. Bounded rationality as proposed by Simon reflects the challenges of making decisions governed by reason and logic under radical uncertainty in which no computable solution is available. Simon is reported to have joked that he should take legal action against his successors who misused his terminology and neglected his insights. 24 *Underline [page 131]:* Economists have adapted the phrase ‘bounded rationality’ to mean something very different from Simon’s description as the consequence of radical uncertainty. They have instead used it to describe the cost of processing information, which then acts as an additional constraint in an optimisation problem. Bounded rationality, in this sense, adds to the optimisation calculation the costs and benefits of obtaining the information which we choose not to have. Of course, this is not what Simon meant *Underline [page 131]:* Bounded rationality as proposed by Simon reflects the challenges of making decisions governed by reason and logic under radical uncertainty in which no computable solution is available. Simon is reported to have joked that he should take legal action against his successors who misused his terminology and neglected his insights. *Highlight [page 131]:* Decisionmakers usually look for the first workable option they can find, not the best option. Since the first option they consider is usually workable, they do not have to generate a large set of options to be sure they get a good one. They generate and evaluate options one at a time and do not bother comparing the advantages and disadvantages of alternatives. 25 *Underline [page 131]:* Decisionmakers usually look for the first workable option they can find, not the best option. Since the first option they consider is usually workable, they do not have to generate a large set of options to be sure they get a good one. They generate and evaluate options one at a time and do not bother comparing the advantages and disadvantages of alternatives. 25 *Highlight [page 132]:* Klein describes the reality of decision-making in complex situations, which require the search for a workable solution rather than a process of optimisation. *Underline [page 132]:* search for a workable solution rather than a process of optimisation. *Highlight [page 132]:* the best is the enemy of the good’. Real people do not optimise, calculate subjective probabilities and maximise expected utilities; not because they are lazy, or do not have the time, but because they know that they cannot conceivably have the information required to engage in such calculation. Nevertheless good decision-makers, like Klein’s firefighters and paramedics, or Warren Buffett or Steve -obs, are rightly respected for their judgement. *Highlight [page 132]:* The group has emphasised the value of simple heuristics – or rules of thumb – in enabling us to resolve situations characterised by radical uncertainty. Gigerenzer and his colleagues have promoted a toolbox comprising ‘fast and frugal’ heuristics. *Highlight [page 133]:* There is an alternative story to that told by behavioural economics. It is that many of the characteristics of human reasoning which behavioural economics describes as biases are in fact adaptive – beneficial to success in the large real worlds in which people live, even if they are sometimes misleading in the small worlds created for the purposes of economic modelling and experimental psychology. It is an account which substitutes evolutionary rationality for axiomatic rationality. *Underline [page 133]:* It is an account which substitutes evolutionary rationality for axiomatic rationality. *Highlight [page 134]:* We cope with the future by organising our lives around reference narratives. These reference narratives are not necessarily worked out in specific detail but they provide a basis for planning and a framework for day-to-day choices. *Highlight [page 136]:* Survival of the fittest was, he argued, inherent in a competitive market. The observation that there are mechanisms of evolution other than the biological, and that competitive markets may be one such evolutionary mechanism, was correct and important. *Highlight [page 139]:* The Socratic dialogue is a long-established method of seeking truth by exposing the competing arguments of protagonists. The objective in all these processes is to find, through group interaction, a narrative to which all can subscribe – and to set a course of future action in the light of that narrative. The observations of participants contribute to that narrative, and the meaning of these observations is derived from the context in which they are made. 13 Evolution gave us a capacity to reason which, as a 2017 book by two French researchers in cognitive science, Hugo Mercier and Dan Sperber, explains, ‘is not geared to solitary use’. 14 Evolution has produced the collective intelligence and social norms and institutions which are ‘the secret of our success’; these social capabilities provide the reason that humans dominate the planet. 15 *Underline [page 139]:* The Socratic dialogue is a long-established method of seeking truth by exposing the competing arguments of protagonists. The objective in all these processes is to find, through group interaction, a narrative to which all can subscribe – and to set a course of future action in the light of that narrative. The observations of participants contribute to that narrative, and the meaning of these observations is derived from the context in which they are made. 13 *Highlight [page 187]:* Scenarios are useful ways of beginning to come to terms with an uncertain future. But to ascribe a probability to any particular scenario is misconceived. We have both had the experience of describing alternative future economic scenarios and being asked, ‘So which do you think is going to happen?’ The questioner is unprepared for the correct answer – ‘I think it very unlikely that any of these scenarios will unfold in the manner I have described’. Scenario planning is a way of ordering thoughts about the future, not of predicting it. *Highlight [page 190]:* Humans are not computers. We make decisions using judgement, instinct and emotions. And when we explain the decisions we have made, either to ourselves or to others, our explanation usually takes narrative form. As David Tuckett, a social scientist and psychoanalyst, has argued, decisions require us ‘to feel sufficiently convinced about the anticipated outcomes to act’. 19 Narratives are the mechanism by which conviction is developed. Narratives underpin our sense of identity, and enable us to recreate decisions of the past and imagine decisions we will face in the future. Emotions and human cognition are not separate processes. In developing his concept of conviction narrative theory, Tuckett suggests that a decisionmaker must manage the ‘emotions evoked during narrative simulation’ in order to develop sufficient conviction about a proposed decision. 20 Tuckett’s thesis was formulated after listening to participants in financial markets describing at length how they did in practice make the decisions. The Reverend Bayes was rarely discussed. *Underline [page 190]:* decisions require us ‘to feel sufficiently convinced about the anticipated outcomes to act’. 19 Narratives are the mechanism by which conviction is developed. *Highlight [page 193]:* In a world in which to list all possible outcomes and their probabilities would be impossibly complex, narratives are an essential part of how we reason. But they are not just a way in which we provide ourselves with the ‘best explanation’. They play a crucial role in how we communicate with each other, and how we reach collective decisions. --- Under uncertainty, you don't maximise expected utility. You maxipok. (?) That's a kind of maximisation!