MIRI Senior Research Fellow Eliezer Yudkowsky has a new book out today: Inadequate Equilibria: Where and How Civilizations Get Stuck, a discussion of societal dysfunction, exploitability, and self-evaluation. From the preface:
Inadequate Equilibria is a book about a generalized notion of efficient markets, and how we can use this notion to guess where society will or won’t be effective at pursuing some widely desired goal.An efficient market is one where smart individuals should generally doubt that they can spot overpriced or underpriced assets. We can ask an analogous question, however, about the “efficiency” of other human endeavors.
Suppose, for example, that someone thinks they can easily build a much better and more profitable social network than Facebook, or easily come up with a new treatment for a widespread medical condition. Should they question whatever clever reasoning led them to that conclusion, in the same way that most smart individuals should question any clever reasoning that causes them to think AAPL stock is underpriced? Should they question whether they can “beat the market” in these areas, or whether they can even spot major in-principle improvements to the status quo? How “efficient,” or adequate, should we expect civilization to be at various tasks?
There will be, as always, good ways and bad ways to reason about these questions; this book is about both.
The book is available from Amazon (in print and Kindle), on iBooks, as a pay-what-you-want digital download, and as a web book at equilibriabook.com. The book has also been posted to Less Wrong 2.0.
The book’s contents are:
A comparison of two “wildly different, nearly cognitively nonoverlapping” approaches to thinking about outperformance: modest epistemology, and inadequacy analysis.
2. An Equilibrium of No Free Energy
How, in principle, can society end up neglecting obvious low-hanging fruit?
Why does our civilization actually end up neglecting low-hanging fruit?
4. Living in an Inadequate World
How can we best take into account civilizational inadequacy in our decision-making?
Three examples of modesty in practical settings.
6. Against Modest Epistemology
An argument against the “epistemological core” of modesty: that we shouldn’t take our own reasoning and meta-reasoning at face value in cases in the face of disagreements or novelties.
7. Status Regulation and Anxious Underconfidence
On causal accounts of modesty.
Although Inadequate Equilibria isn’t about AI, I consider it one of MIRI’s most important nontechnical publications to date, as it helps explain some of the most basic tools and background models we use when we evaluate how promising a potential project, research program, or general strategy is.
