The basic reasons I expect AGI ruin

Posted by & filed under Analysis.

I’ve been citing AGI Ruin: A List of Lethalities to explain why the situation with AI looks lethally dangerous to me. But that post is relatively long, and emphasizes specific open technical problems over “the basics”. Here are 10 things I’d focus on if I were giving “the basics” on why I’m so worried:[1] 1…. Read more »

Yudkowsky on AGI risk on the Bankless podcast

Posted by & filed under Analysis, Conversations.

Eliezer gave a very frank overview of his take on AI two weeks ago on the cryptocurrency show Bankless:  I’ve posted a transcript of the show and a follow-up Q&A below. Thanks to Andrea_Miotti, remember, and vonk for help posting transcripts. Intro Eliezer Yudkowsky: [clip] I think that we are hearing the last winds start… Read more »

July 2022 Newsletter

Posted by & filed under Newsletters.

MIRI has put out three major new posts: AGI Ruin: A List of Lethalities. Eliezer Yudkowsky lists reasons AGI appears likely to cause an existential catastrophe, and reasons why he thinks the current research community—MIRI included—isn't succeeding at preventing this from happening A central AI alignment problem: capabilities generalization, and the sharp left turn. Nate Soares describes… Read more »

Shah and Yudkowsky on alignment failures

Posted by & filed under Analysis, Conversations.

  This is the final discussion log in the Late 2021 MIRI Conversations sequence, featuring Rohin Shah and Eliezer Yudkowsky, with additional comments from Rob Bensinger, Nate Soares, Richard Ngo, and Jaan Tallinn. The discussion begins with summaries and comments on Richard and Eliezer’s debate. Rohin’s summary has since been revised and published in the… Read more »

Ngo and Yudkowsky on scientific reasoning and pivotal acts

Posted by & filed under Analysis, Conversations.

This is a transcript of a conversation between Richard Ngo and Eliezer Yudkowsky, facilitated by Nate Soares (and with some comments from Carl Shulman). This transcript continues the Late 2021 MIRI Conversations sequence, following Ngo’s view on alignment difficulty.   Color key:  Chat by Richard and Eliezer   Other chat      14. October 4 conversation… Read more »

February 2022 Newsletter

Posted by & filed under Newsletters.

As of yesterday, we've released the final posts in the Late 2021 MIRI Conversations sequence, a collection of (relatively raw and unedited) AI strategy conversations: Ngo's view on alignment difficulty Ngo and Yudkowsky on scientific reasoning and pivotal acts Christiano and Yudkowsky on AI predictions and human intelligence Shah and Yudkowsky on alignment failures Eliezer Yudkowsky, Nate… Read more »

January 2022 Newsletter

Posted by & filed under Newsletters.

MIRI updates MIRI's $1.2 million Visible Thoughts Project bounty now has an FAQ, and an example of a successful partial run that you can use to inform your own runs. Scott Alexander reviews the first part of the Yudkowsky/Ngo debate. See also Richard Ngo's reply, and Rohin Shah's review of several posts from the Late 2021 MIRI Conversations. From Evan… Read more »