Posts By: Rob Bensinger

MIRI updates Abram Demski distinguishes different versions of the problem of “pointing at” human values in AI alignment. Evan Hubinger discusses “Risks from Learned Optimization” on the AI X-Risk Research Podcast. Eliezer Yudkowsky comments on AI safety via debate and Goodhart’s law. MIRI supporters donated ~$135k on Giving Tuesday, of which ~26% was matched by Facebook and ~28% by employers… Read more »

MIRI updates MIRI’s Evan Hubinger uses a notion of optimization power to define whether AI systems are compatible with the strategy-stealing assumption. MIRI’s Abram Demski discusses debate approaches to AI safety that don’t rely on factored cognition. Evan argues that the first AGI systems are likely to be very similar to each other, and discusses… Read more »

MIRI COO Malo Bourgon reviews our past year and discusses our future plans in 2020 Updates and Strategy. Our biggest update is that we've made less concrete progress than we expected on the new research we described in 2018 Update: Our New Research Directions. As a consequence, we're scaling back our work on these research… Read more »

MIRI researcher Scott Garrabrant has completed his Cartesian Frames sequence. Scott also covers the first two posts' contents in video form. Other MIRI updates Contrary to my previous announcement, MIRI won’t be running a formal fundraiser this year, though we’ll still be participating in Giving Tuesday and other matching opportunities. To donate and get information on tax-advantaged… Read more »

Starting today, Scott Garrabrant has begun posting Cartesian Frames, a sequence introducing a new conceptual framework Scott has found valuable for thinking about agency. In Scott's words: Cartesian Frames are “applying reasoning like Pearl's to objects like game theory's, with a motivation like Hutter's”. Scott will be giving an online talk introducing Cartesian frames this… Read more »

Abram Demski and Scott Garrabrant have made a major update to "Embedded Agency", with new discussions of ε-exploration, Newcomblike problems, reflective oracles, logical uncertainty, Goodhart's law, and predicting rare catastrophes, among other topics. Abram has also written an overview of what good reasoning looks in the absence of Bayesian updating: Radical Probabilism. One recurring theme: [I]n general… Read more »

MIRI updates Three questions from MIRI's Abram Demski: What does it mean to apply decision theory?, How “honest” is GPT-3?, and How should AI debate be judged? A transcript from MIRI researcher Scott Garrabrant: What Would I Do? Self-Prediction in Simple Algorithms. MIRI researcher Buck Shlegeris reviews the debate on what the history of nuclear weapons implies… Read more »

After completing a study fellowship at MIRI that he began in late 2019, Blake Jones is joining the MIRI research team full-time! Blake joins MIRI after a long career working on low-level software systems such as the Solaris operating system and the Oracle database. Other MIRI updates MIRI researcher Evan Hubinger goes on the FLI… Read more »