Rationality - Peter's notes

I like [Yudkowsky’s characterisation](https://www.lesswrong.com/posts/4ARtkT3EYox3THYjF/rationality-is-systematized-winning): 1. Epistemic rationality is about reliable belief formation. 2. Instrumental rationality is about effective action. [1] Both kinds of rationality often involve heavy reliance on common sense, intuition, traditions, norms and deference to expertise. Our individual cognitive resources are limited, so we must employ the resources of others, take shortcuts and so on—we cannot do all the work ourselves. Thankfully, other people can often be trusted, and most of the time, “good enough” is actually good enough. Only rarely does instrumental rationality justifiably involve the arduous and hazardous task of “explicitly thinking things through for oneself”; perhaps never is it fully articulate or intuition free. Yudkowsky writes: > There is a meme which says that a certain ritual of cognition is the paragon of /reasonableness/ and so defines what the /reasonable/ people do. But alas, the /reasonable/ people often get their butts handed to them by the /unreasonable/ ones, because the universe isn’t always /reasonable/. > > [Explicit reasoning] is just /a/ way of doing things, not necessarily /the/most [formidable](https://www.lesswrong.com/lw/2c/a_sense_that_more_is_possible/) ; it is how professors talk to each other in debate halls, which sometimes works, and sometimes doesn’t. If a hoard of barbarians attacks the debate hall, the truly prudent and flexible agent will abandon reasonableness. Does rationality necessarily involve wanting what is good? As I understand it, the Kantian conception says “yes”, while the Humean naturalistic conception say “no”. I recently heard someone (Lucas Perry, I think) say that views on this question tightly correlate with views on moral realism, which rings true to me. A point which should be obvious: rationality is not a binary property that minds either have or do not have, or that all humans share equally. It is a capability that is cultivated, developed and refined over a lifetime, like other capabilities. Rationality is a loaded term with lots of confusing associations (c.f. [The Straw Vulcan](https://tvtropes.org/pmwiki/pmwiki.php/Main/StrawVulcan); rationalism in philosophy). I think “effective action” may be a better phrase to use than “instrumental rationality”; I’ll try it below. Effective action is not about a particular method. It involves awareness that there are many methods, and an ability to select those which are most appropriate for the situation. Often, especially for important decisions, it means bringing several methods to bear and using them to generate an overall judgement. A multi-method approach lets us explore a broader range of solution space, then triangulate and cross-check solutions as we find them. Sometimes, a solution that looks great from one perspective looks fatally flawed from another (e.g. XXX); other times, an extra perspective reveals a much cheaper solution than any of the others (e.g. XXX). ## Aside: Articulation, and “showing your work” Epistemic and instrumental rationality always involves a great deal that is inarticulate, unspoken, intuitive, unconscious. Even with the most explicit and articulate methods, what is articulated is just the tip of the iceberg of cognitive process that leads to reasonable action. (I took Nick Chater’s “The Mind is Flat” course some years ago: so far as I understand his story, it is compatible with picture in this paragraph.) Upshot: we should not assume that someone who acts rationally will be able to fully articulate their reasoning or otherwise “show their work”. They may tell a compelling story, but it won’t be the full story, because the fully story is very, very complicated. That said: willingness and ability to articulate reasons may correlate with effective action in at least some domains. I’m disturbingly unsure about where and how often this is true, but I’m also wary of romanticism on this point. Some beliefs are hard to put into practice until you can articulate them. [1] The pragmatist in me wonders about the epistemic/instrumental distinction: can these really be teased apart? Insofar as they can, is it useful to think this way? #todo ## Notes * I want to distinguish explicit articulation from reasoning. * The dual purpose of explicit articulation (social cognition, social cohesion) can be confusing. * If you’re mostly interested in cognition rather than cohesion you’re going to want to create an exceptionally “safe space” where people do not feel pressure to signal loyalty. * Maybe things get particularly hairy with moral philosophy, where it’s unclear whether we’re doing truth seeking or truth construction, and normative disagreement is reputationally high stakes. * Bring in the CEO and press secretary stuff * Individual, social, * Explicit, implicit * Rhetoric poetry philosophy Plato * Wisdom