November 2017 Newsletter

Posted by & filed under Newsletters.

Eliezer Yudkowsky has written a new book on civilizational dysfunction and outperformance: Inadequate Equilibria: Where and How Civilizations Get Stuck. The full book will be available in print and electronic formats November 16. To preorder the ebook or sign up for updates, visit We’re posting the full contents online in stages over the next… Read more »

October 2017 Newsletter

Posted by & filed under Newsletters.

“So far as I can presently estimate, now that we’ve had AlphaGo and a couple of other maybe/maybe-not shots across the bow, and seen a huge explosion of effort invested into machine learning and an enormous flood of papers, we are probably going to occupy our present epistemic state until very near the end. “[…I]t’s hard… Read more »

September 2017 Newsletter

Posted by & filed under Newsletters.

Research updates “Incorrigibility in the CIRL Framework”: a new paper by MIRI assistant researcher Ryan Carey responds to Hadfield-Menell et al.’s “The Off-Switch Game”. New at IAFF: The Three Levels of Goodhart’s Curse; Conditioning on Conditionals; Stable Pointers to Value: An Agent Embedded in Its Own Utility Function; Density Zero Exploration; Autopoietic Systems and the Difficulty… Read more »

August 2017 Newsletter

Posted by & filed under Newsletters.

Research updates “A Formal Approach to the Problem of Logical Non-Omniscience”: We presented our work on logical induction at the 16th Conference on Theoretical Aspects of Rationality and Knowledge. New at IAFF: Smoking Lesion Steelman; “Like This World, But…”; Jessica Taylor’s Current Thoughts on Paul Christiano’s Research Agenda; Open Problems Regarding Counterfactuals: An Introduction For Beginners “A Game-Theoretic… Read more »

July 2017 Newsletter

Posted by & filed under Newsletters.

A number of major mid-year MIRI updates: we received our largest donation to date, $1.01 million from an Ethereum investor! Our research priorities have also shifted somewhat, reflecting the addition of four new full-time researchers (Marcello Herreshoff, Sam Eisenstat, Tsvi Benson-Tilsen, and Abram Demski) and the departure of Patrick LaVictoire and Jessica Taylor. Research updates New at IAFF:… Read more »

June 2017 Newsletter

Posted by & filed under Newsletters.

Research updates A new AI Impacts paper: “When Will AI Exceed Human Performance?” News coverage at Digital Trends and MIT Technology Review. New at IAFF: Cooperative Oracles; Jessica Taylor on the AAMLS Agenda; An Approach to Logically Updateless Decisions Our 2014 technical agenda, “Agent Foundations for Aligning Machine Intelligence with Human Interests,” is now available as a… Read more »

May 2017 Newsletter

Posted by & filed under Newsletters.

Research updates New at IAFF: The Ubiquitous Converse Lawvere Problem; Two Major Obstacles for Logical Inductor Decision Theory; Generalizing Foundations of Decision Theory II. New at AI Impacts: Guide to Pages on AI Timeline Predictions “Decisions Are For Making Bad Outcomes Inconsistent”: Nate Soares dialogues on some of the deeper issues raised by our “Cheating Death in… Read more »

2017 Updates and Strategy

Posted by & filed under MIRI Strategy.

In our last strategy update (August 2016), Nate wrote that MIRI’s priorities were to make progress on our agent foundations agenda and begin work on our new “Alignment for Advanced Machine Learning Systems” agenda, to collaborate and communicate with other researchers, and to grow our research and ops teams. Since then, senior staff at MIRI… Read more »