February 2016 Newsletter

Posted by & filed under Newsletters.

Research updates New at IAFF: Thoughts on Logical Dutch Book Arguments; Another View of Quantilizers: Avoiding Goodhart’s Law; Another Concise Open Problem General updates Fundraiser and grant successes: MIRI will be working with AI pioneer Stuart Russell and a to-be-determined postdoctoral researcher on the problem of corrigibility, thanks to a $75,000 grant by the Center… Read more »

January 2016 Newsletter

Posted by & filed under Newsletters.

Research updates A new paper: “Proof-Producing Reflection for HOL” A new analysis: Safety Engineering, Target Selection, and Alignment Theory New at IAFF: What Do We Need Value Learning For?; Strict Dominance for the Modified Demski Prior; Reflective Probability Distributions and Standard Models of Arithmetic; Existence of Distributions That Are Expectation-Reflective and Know It; Concise Open… Read more »

The need to scale MIRI’s methods

Posted by & filed under Analysis.

Andrew Critch, one of the new additions to MIRI’s research team, has taken the opportunity of MIRI’s winter fundraiser to write on his personal blog about why he considers MIRI’s work important. Some excerpts: Since a team of CFAR alumni banded together to form the Future of Life Institute (FLI), organized an AI safety conference… Read more »

New paper: “Proof-producing reflection for HOL”

Posted by & filed under Papers.

MIRI Research Fellow Benya Fallenstein and Research Associate Ramana Kumar have co-authored a new paper on machine reflection, “Proof-producing reflection for HOL with an application to model polymorphism.” HOL stands for Higher Order Logic, here referring to a popular family of proof assistants based on Church’s type theory. Kumar and collaborators have previously formalized within… Read more »

December 2015 Newsletter

Posted by & filed under Newsletters.

Research updates New papers: “Formalizing Convergent Instrumental Goals” and “Quantilizers: A Safer Alternative to Maximizers for Limited Optimization.” These papers have been accepted to the AAAI-16 workshop on AI, Ethics and Society. New at AI Impacts: Recently at AI Impacts New at IAFF: A First Look at the Hard Problem of Corrigibility; Superrationality in Arbitrary… Read more »

New paper: “Quantilizers”

Posted by & filed under Papers.

MIRI Research Fellow Jessica Taylor has written a new paper on an error-tolerant framework for software agents, “Quantilizers: A safer alternative to maximizers for limited optimization.” Taylor’s paper will be presented at the AAAI-16 AI, Ethics and Society workshop. The abstract reads: In the field of AI, expected utility maximizers are commonly used as a… Read more »

New paper: “Formalizing convergent instrumental goals”

Posted by & filed under Papers.

Tsvi Benson-Tilsen, a MIRI associate and UC Berkeley PhD candidate, has written a paper with contributions from MIRI Executive Director Nate Soares on strategies that will tend to be useful for most possible ends: “Formalizing convergent instrumental goals.” The paper will be presented as a poster at the AAAI-16 AI, Ethics and Society workshop. Steve… Read more »

November 2015 Newsletter

Posted by & filed under Newsletters.

Research updates A new paper: Leó Szilárd and the Danger of Nuclear Weapons New at IAFF: Subsequence Induction A shortened version of the Reflective Oracles paper has been published in the LORI 2015 conference proceedings. General updates Castify has released professionally recorded audio versions of Eliezer Yudkowsky’s Rationality: From AI to Zombies: Part 1, Part… Read more »