March 2021 Newsletter

April 1, 2021 | Rob Bensinger | Newsletters

MIRI updates

MIRI's Eliezer Yudkowsky and Evan Hubinger comment in some detail on Ajeya Cotra's The Case for Aligning Narrowly Superhuman Models. This conversation touches on some of the more important alignment research views at MIRI, such as the view that alignment requires a thorough understanding of AGI systems' reasoning "under the hood", and the view that early AGI systems should most likely avoid human modeling if possible.
From Eliezer Yudkowsky: A Semitechnical Introductory Dialogue on Solomonoff Induction. (Also discussed by Richard Ngo.)
MIRI research associate Vanessa Kosoy discusses infra-Bayesianism on the AI X-Risk Research Podcast.
Eliezer Yudkowsky and Chris Olah discuss ML transparency on social media.

News and links

Brian Christian, author of The Alignment Problem: Machine Learning and Human Values, discusses his book on the 80,000 Hours Podcast.
Chris Olah's team releases Multimodal Neurons in Artificial Neural Networks, on artificial neurons that fire for multiple conceptually related stimuli.
Vitalik Buterin reflects on Inadequate Equilibria's arguments in the course of discussing prediction market inefficiencies.

February 2021 Newsletter

March 2, 2021 | Rob Bensinger | Newsletters

MIRI updates

Abram Demski distinguishes different versions of the problem of “pointing at” human values in AI alignment.
Evan Hubinger discusses “Risks from Learned Optimization” on the AI X-Risk Research Podcast.
Eliezer Yudkowsky comments on AI safety via debate and Goodhart’s law.
MIRI supporters donated ~$135k on Giving Tuesday, of which ~26% was matched by Facebook and ~28% by employers for a total of $207,436! MIRI also received $6,624 from TisBest Philanthropy in late December, largely through Round Two of Ray Dalio’s #RedefineGifting initiative. Our thanks to all of you!
Spencer Greenberg discusses society and education with Anna Salamon and Duncan Sabien on the Clearer Thinking podcast.
We Want MoR: Eliezer participates in a (spoiler-laden) discussion of Harry Potter and the Methods of Rationality.

News and links

Richard Ngo reflects on his time in effective altruism:

[…] Until recently, I was relatively passive in making big decisions. Often that meant just picking the most high-prestige default option, rather than making a specific long-term plan. This also involved me thinking about EA from a “consumer” mindset rather than a “producer” mindset. When it seemed like something was missing, I used to wonder why the people responsible hadn’t done it; now I also ask why I haven’t done it, and consider taking responsibility myself.

Partly that’s just because I’ve now been involved in EA for longer. But I think I also used to overestimate how established and organised EA is. In fact, we’re an incredibly young movement, and we’re still making up a lot of things as we go along. That makes proactivity more important.

Another reason to value proactivity highly is that taking the most standard route to success is often overrated. […] My inspiration in this regard is a friend of mine who has, three times in a row, reached out to an organisation she wanted to work for and convinced them to create a new position for her.
Ngo distinguishes claims about goal specification, orthogonality, instrumental convergence, value fragility, and Goodhart’s law based on whether they refer to systems at training time versus deployment time.
Connor Leahy, author of The Hacker Learns to Trust, argues (among other things) that “GPT-3 is our last warning shot” for coordinating to address AGI alignment. (Podcast version.) I include this talk because it’s a good talk and the topic warrants discussion, though MIRI staff don’t necessarily endorse this claim — and Eliezer would certainly object to any claim that something is a fire alarm for AGI.
OpenAI safety researchers including Dario Amodei, Paul Christiano, and Chris Olah depart OpenAI.
OpenAI’s DALL-E uses GPT-3 for image generation, while CLIP exhibits impressive zero-shot image classification capabilities. Gwern Branwen comments in his newsletter.

January 2021 Newsletter

January 27, 2021 | Rob Bensinger | Newsletters

December 2020 Newsletter

December 30, 2020 | Rob Bensinger | Newsletters

MIRI COO Malo Bourgon reviews our past year and discusses our future plans in 2020 Updates and Strategy.

Our biggest update is that we've made less concrete progress than we expected on the new research we described in 2018 Update: Our New Research Directions. As a consequence, we're scaling back our work on these research directions, and looking for new angles of attack that have better odds of resulting in a solution to the alignment problem.

Other MIRI updates

A new paper from MIRI researcher Evan Hubinger: "An Overview of 11 Proposals for Building Safe Advanced AI."
A belated paper announcement from last year: Andrew Critch's "A Parametric, Resource-Bounded Generalization of Löb's Theorem, and a Robust Cooperation Criterion for Open-Source Game Theory", a result originally written up during his time at MIRI, has been published in the Journal of Symbolic Logic.
MIRI's Abram Demski introduces Learning Normativity: A Research Agenda. See also Abram's new write-up, Normativity.
Evan Hubinger clarifies inner alignment terminology.
The Survival and Flourishing Fund (SFF) has awarded MIRI $563,000 in its latest round of grants! Our enormous gratitude to SFF's grant recommenders and funders.
A Map That Reflects the Territory is a new print book set collecting the top LessWrong essays of 2018, including essays by MIRI researchers Eliezer Yudkowsky, Abram Demski, and Scott Garrabrant.
DeepMind's Rohin Shah gives his overview of Scott Garrabrant's Cartesian Frames framework.

News and links

Daniel Filan launches the AI X-Risk Research Podcast (AXRP) with episodes featuring Adam Gleave, Rohin Shah, and Andrew Critch.
DeepMind's AlphaFold represents a very large advance in protein structure prediction.
Metaculus launches Forecasting AI Progress, an open four-month tournament to predict advances in AI, with a $50,000 prize pool.
Continuing the Takeoffs Debate: Richard Ngo responds to Paul Christiano's "changing selection pressures" argument against hard takeoff.
OpenAI's Beth Barnes discusses the obfuscated arguments problem for AI safety via debate:

Previously we hoped that debate/IDA could verify any knowledge for which such human-understandable arguments exist, even if these arguments are intractably large. We hoped the debaters could strategically traverse small parts of the implicit large argument tree and thereby show that the whole tree could be trusted.

The obfuscated argument problem suggests that we may not be able to rely on debaters to find flaws in large arguments, so that we can only trust arguments when we could find flaws by recursing randomly—e.g. because the argument is small enough that we could find a single flaw if one existed, or because the argument is robust enough that it is correct unless it has many flaws.
Some AI Research Areas and Their Relevance to Existential Safety: Andrew Critch compares out-of-distribution robustness, agent foundations, multi-agent RL, preference learning, and other research areas.
Ben Hoskin releases his 2020 AI Alignment Literature Review and Charity Comparison.
Open Philanthropy summarizes its AI governance grantmaking to date.

2020 Updates and Strategy

December 21, 2020 | Malo Bourgon | MIRI Strategy, News

MIRI’s 2020 has been a year of experimentation and adjustment. In response to the COVID-19 pandemic, we largely moved our operations to more rural areas in March, and shifted to a greater emphasis on remote work. We took the opportunity to try new work set-ups and approaches to research, and have been largely happy with the results.

At the same time, 2020 saw limited progress in the research MIRI’s leadership had previously been most excited about: the new research directions we started in 2017. Given our slow progress to date, we are considering a number of possible changes to our strategy, and MIRI’s research leadership is shifting much of their focus toward searching for more promising paths.

November 2020 Newsletter

November 30, 2020 | Rob Bensinger | Newsletters

October 2020 Newsletter

October 23, 2020 | Rob Bensinger | Newsletters

April 2021 Newsletter

March 2021 Newsletter

February 2021 Newsletter

January 2021 Newsletter

December 2020 Newsletter

2020 Updates and Strategy

November 2020 Newsletter

October 2020 Newsletter

Search

Browse

Subscribe