January 2016 Newsletter

Research updates

• A new paper: “Proof-Producing Reflection for HOL”
• A new analysis: Safety Engineering, Target Selection, and Alignment Theory
• New at IAFF: What Do We Need Value Learning For?; Strict Dominance for the Modified Demski Prior; Reflective Probability Distributions and Standard Models of Arithmetic; Existence of Distributions That Are Expectation-Reflective and Know It; Concise Open Problem in Logical Uncertainty

General updates

• Our Winter Fundraiser is over! A total of 176 people donated $351,411, including some surprise matching donors. All of you have our sincere thanks.
• Jed McCaleb writes on why MIRI matters, while Andrew Critch writes on the need to scale MIRI’s methods.
• We attended NIPS, which hosted a symposium on the “social impacts of machine learning” this year. Viktoriya Krakovna summarizes her impressions.
• We’ve moved to a new, larger office with the Center for Applied Rationality (CFAR), a few floors up from our old one.
• Our paper announcements now have their own MIRI Blog category.

News and links

• “The 21st Century Philosophers”: AI safety research gets covered in OZY.
• Sam Altman and Elon Musk have brought together leading AI researchers to form a new $1 billion nonprofit, OpenAI. Andrej Karpathy explains OpenAI’s plans (link), and Altman and Musk provide additional background (link).
• Alphabet chairman Eric Schmidt and Google Ideas director Jared Cohen write on the need to “establish best practices to avoid undesirable outcomes” from AI.
• A new Future of Humanity Institute (FHI) paper: “Learning the Preferences of Ignorant, Inconsistent Agents.”
• Luke Muehlhauser and The Telegraph signal-boost FHI’s AI safety job postings (deadline Jan. 6). The Global Priorities Project is also seeking summer interns (deadline Jan. 10).
• CFAR is running a matching fundraiser through the end of January.