Updates Our 2019 fundraiser ended Dec. 31. We'll have more to say in a few weeks in our fundraiser retrospective, but for now, a big thank you to the ~240 donors who together donated more than $526,000, including $67,484 in the first 20 seconds of Giving Tuesday (not counting matching dollars, which have yet to be announced). Jan. 15… Read more »
Posts By: Rob Bensinger
December 2019 Newsletter
From now through the end of December, MIRI's 2019 Fundraiser is live! See our fundraiser post for updates on our past year and future plans. One of our biggest updates, I'm happy to announce, is that we've hired five new research staff, with a sixth to join us in February. For details, see Workshops and… Read more »
November 2019 Newsletter
I'm happy to announce that Nate Soares and Ben Levinstein's “Cheating Death in Damascus” has been accepted for publication in The Journal of Philosophy (previously voted the second-highest-quality journal in philosophy). In other news, MIRI researcher Buck Shlegeris has written over 12,000 words on a variety of MIRI-relevant topics in an EA Forum AMA. (Example topics: advice… Read more »
October 2019 Newsletter
Updates Ben Pace summarizes a second round of AI Alignment Writing Day posts. The Zettelkasten Method: MIRI researcher Abram Demski describes a note-taking system that's had a large positive effect on his research productivity. Will MacAskill writes a detailed critique of functional decision theory; Abram Demski (1, 2) and Matthew Graves respond in the comments. News and… Read more »
September 2019 Newsletter
Updates We ran a very successful MIRI Summer Fellows Program, which included a day where participants publicly wrote up their thoughts on various AI safety topics. See Ben Pace’s first post in a series of roundups. A few highlights from the writing day: Adele Lopez's Optimization Provenance; Daniel Kokotajlo's Soft Takeoff Can Still Lead to Decisive Strategic Advantage and The… Read more »
August 2019 Newsletter
Updates MIRI research associate Stuart Armstrong is offering $1000 for good questions to ask an Oracle AI. Recent AI safety posts from Stuart: Indifference: Multiple Changes, Multiple Agents; Intertheoretic Utility Comparison: Examples; Normalising Utility as Willingness to Pay; and Partial Preferences Revisited. MIRI researcher Buck Shlegeris has put together a quick and informal AI safety reading… Read more »
July 2019 Newsletter
Hubinger et al.'s “Risks from Learned Optimization in Advanced Machine Learning Systems”, one of our new core resources on the alignment problem, is now available on arXiv, the AI Alignment Forum, and LessWrong. In other news, we received an Ethereum donation worth $230,910 from Vitalik Buterin — the inventor and co-founder of Ethereum, and now our third-largest… Read more »
New paper: “Risks from learned optimization”
Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, and Scott Garrabrant have a new paper out: “Risks from learned optimization in advanced machine learning systems.” The paper’s abstract: We analyze the type of learned optimization that occurs when a learned model (such as a neural network) is itself an optimizer—a situation we refer to… Read more »
