MIRI is offering $200,000 to build a dataset of AI-dungeon-style writing annotated with the thoughts used in the writing process, and an additional $1,000,000 for scaling that dataset an additional 10x: the Visible Thoughts Project.
Additionally, MIRI is in the process of releasing a series of chat logs, the Late 2021 MIRI Conversations, featuring relatively unedited and raw conversations between Eliezer Yudkowsky, Richard Ngo, Paul Christiano, and a number of other AI x-risk researchers.
As background, we've also released an anonymous discussion with Eliezer Yudkowsky on AGI interventions (cf. Zvi Mowshowitz's summary) and Nate Soares' comments on Carlsmith’s “Is power-seeking AI an existential risk?” (one of several public reviews of Carlsmith's report).
The logs so far:
- Ngo and Yudkowsky on alignment difficulty — A pair of opening conversations asking how easy it is to avoid "consequentialism" in powerful AGI systems.
- Ngo and Yudkowsky on AI capability gains — Richard and Eliezer continue their dialogue.
- Yudkowsky and Christiano discuss "Takeoff Speeds" — Paul Christiano joins the conversation, and debates hard vs. soft takeoff with Eliezer.
- Soares, Tallinn, and Yudkowsky discuss AGI cognition — Jaan Tallinn and Nate Soares weigh in on the conversation so far.
- Christiano, Cotra, and Yudkowsky on AI progress — Paul and Eliezer begin a longer AGI forecasting discussion, joined by Ajeya Cotra.
- Shulman and Yudkowsky on AI progress — Carl Shulman weighs in on the Paul/Eliezer/Ajeya conversation.
- More Christiano, Cotra, and Yudkowsky on AI progress — A discussion of “why should we expect early prototypes to be low-impact?”, and of concrete predictions.
- Conversation on technology forecasting and gradualism — A larger-group discussion, following up on the Paul/Eliezer debate.
Eliezer additionally wrote a dialogue, Biology-Inspired AGI Timelines: The Trick That Never Works, which Holden Karnofsky responded to.
News and links
- How To Get Into Independent Research On Alignment/Agency: John Wentworth gives an excellent overview of how to get started doing AI alignment research.
- A new summer fellowship, Principles of Intelligent Behavior in Biological and Social Systems, is seeking applicants to spend three months in 2022 working on AI alignment "through studying analogies to many complex systems (evolution, brains, language, social structures…)". Apply by January 16.
