[Rationality: From AI to Zombies](https://intelligence.org/rationality-ai-zombies/) is one of the best things I’ve read on how to think well. It is very long, but very worth it. If you find the style off-putting… I don’t have a quick alternative recommendation, unfortunately. This note is a stub: a reminder to myself to write more soon. #todo https://www.lesswrong.com/s/7gRSERQZbqTuLX5re -- fake beliefs series https://www.lesswrong.com/posts/NMoLJuDJEms7Ku9XS/guessing-the-teacher-s-password https://www.lesswrong.com/posts/a7n8GdKiAZRX86T5A/making-beliefs-pay-rent-in-anticipated-experiences > One principle of rationality is that "[beliefs should pay rent in **anticipated experiences**](https://www.lesswrong.com/s/7gRSERQZbqTuLX5re/p/a7n8GdKiAZRX86T5A)." If you believe in something, what do you expect to be different as a result? What does the belief say should happen, and what does it say should _not_ happen? If you have a verbal disagreement with someone, how does your disagreement cash out in differing expectations? https://www.lesswrong.com/posts/dLbkrPu5STNCBLRjr/applause-lights https://www.lesswrong.com/posts/CqyJzDZWvGhhFJ7dY/belief-in-belief > You can much more easily believe that it is _proper_, that it is _good_ and _virtuous_ and _beneficial_, to believe that the Ultimate Cosmic Sky is both perfectly blue and perfectly green. Dennett calls this “belief in belief.”1 > But there are also less explicit forms of belief in belief. Maybe the dragon-claimant fears the public ridicule that they imagine will result if they publicly confess they were wrong.2 Maybe the dragon-claimant flinches away from the prospect of admitting to themselves that there is no dragon, because it conflicts with their self-image as the glorious discoverer of the dragon, who saw in their garage what all others had failed to see. > If all our thoughts were deliberate verbal sentences like philosophers manipulate, the human mind would be a great deal easier for humans to understand. Fleeting mental images, unspoken flinches, desires acted upon without acknowledgement—these account for as much of ourselves as words. > I still think that Dennett’s notion of _belief in belief_ is the key insight necessary to understand the dragon-claimant. But we need a wider concept of _belief_, not limited to verbal sentences. “Belief” should include unspoken anticipation-controllers. “Belief in belief” should include unspoken cognitive-behavior-guiders. It is not psychologically realistic to say, “The dragon-claimant does not believe there is a dragon in their garage; they believe it is beneficial to believe there is a dragon in their garage.” But it is realistic to say the dragon-claimant _anticipates as if_ there is no dragon in their garage, and _makes excuses as if_ they believed in the belief. #todo re-read my R:AIZ notes