Updates
- MIRI research associate Stuart Armstrong is offering $1000 for good questions to ask an Oracle AI.
- Recent AI safety posts from Stuart: Indifference: Multiple Changes, Multiple Agents; Intertheoretic Utility Comparison: Examples; Normalising Utility as Willingness to Pay; and Partial Preferences Revisited.
- MIRI researcher Buck Shlegeris has put together a quick and informal AI safety reading list.
- There's No Fire Alarm for AGI reports on a researcher's January 2017 prediction that “in the next two years, we will not get 80, 90%” on Winograd schemas, an NLP test. Although this prediction was correct, researchers at Microsoft, Carnegie Mellon and Google Brain, and Facebook have now (2.5 years later) achieved Winograd scores of 89.0 and 90.4.
- Ortega et al.'s “Meta-Learning of Sequential Strategies” includes a discussion of mesa-optimization, independent of Hubinger et al.'s “Risks from Learned Optimization in Advanced Machine Learning Systems,” under the heading of “spontaneous meta-learning.”
News and links
- Wei Dai outlines forum participation as a research strategy.
- On a related note, the posts on the AI Alignment Forum this month were very good — I'll spotlight them all this time around. Dai wrote on the purposes of decision theory research; Shah on learning biases and rewards simultaneously; Kovarik on AI safety debate and its applications; Steiner on the Armstrong agenda and the intentional stance; Trazzi on manipulative AI; Cohen on IRL and imitation; and Manheim on optimizing and Goodhart effects (1, 2, 3).
- Jade Leung discusses AI governance on the AI Alignment Podcast.
- CMU and Facebook researchers' Pluribus program beats human poker professionals — using only $144 in compute. The developers also choose not to release the code: “Because poker is played commercially, the risk associated with releasing the code outweighs the benefits.”
- Microsoft invests $1 billion in OpenAI. From Microsoft's press release: “Through this partnership, the companies will accelerate breakthroughs in AI and power OpenAI’s efforts to create artificial general intelligence (AGI).” OpenAI has also released a paper on “The Role of Cooperation in Responsible AI Development.”
- Ought has a new preferred introduction to their work. See also Paul Christiano's Ought: Why it Matters and Ways to Help.
- FHI has 11 open research positions; applications are due by Aug. 16. You can also apply to CSER's AGI risk research associate position through Aug. 26.
