MIRI’s research to date has focused on the problems that we laid out in our late 2014 research agenda, and in particular on formalizing optimal reasoning for bounded, reflective decision-theoretic agents embedded in their environment. Our research team has since grown considerably, and we have made substantial progress on this agenda, including a major breakthrough… Read more »
Posts By: Rob Bensinger
July 2016 Newsletter
Research updates A new paper: “A Formal Solution to the Grain of Truth Problem.” The paper was presented at UAI-16, and describes the first general reduction of game-theoretic reasoning to expected utility maximization. Participants in MIRI’s recently-concluded Colloquium Series on Robust and Beneficial AI (CSRBAI) have put together AI safety environments for the OpenAI Reinforcement Learning Gym. Help is welcome creating more… Read more »
New paper: “A formal solution to the grain of truth problem”
Future of Humanity Institute Research Fellow Jan Leike and MIRI Research Fellows Jessica Taylor and Benya Fallenstein have just presented new results at UAI 2016 that resolve a longstanding open problem in game theory: “A formal solution to the grain of truth problem.” Game theorists have techniques for specifying agents that eventually do well on… Read more »
June 2016 Newsletter
Research updates New paper: “Safely Interruptible Agents.” The paper will be presented at UAI-16, and is a collaboration between Laurent Orseau of Google DeepMind and Stuart Armstrong of the Future of Humanity Institute (FHI) and MIRI; see FHI’s press release. The paper has received (often hyperbolic) coverage from a number of press outlets, including Business… Read more »
New paper: “Safely interruptible agents”
Google DeepMind Research Scientist Laurent Orseau and MIRI Research Associate Stuart Armstrong have written a new paper on error-tolerant agent designs, “Safely interruptible agents.” The paper is forthcoming at the 32nd Conference on Uncertainty in Artificial Intelligence. Abstract: Reinforcement learning agents interacting with a complex environment like the real world are unlikely to behave optimally… Read more »
May 2016 Newsletter
Research updates Two new papers split logical uncertainty into two distinct subproblems: “Uniform Coherence” and “Asymptotic Convergence in Online Learning with Unbounded Delays.” New at IAFF: An Approach to the Agent Simulates Predictor Problem; Games for Factoring Out Variables; Time Hierarchy Theorems for Distributional Estimation Problems We will be presenting “The Value Learning Problem” at… Read more »
April 2016 Newsletter
Research updates A new paper: “Parametric Bounded Löb’s Theorem and Robust Cooperation of Bounded Agents“ New at IAFF: What Does it Mean for Correct Operation to Rely on Transfer Learning?; Virtual Models of Virtual AIs in Virtual Worlds General updates We’re currently accepting applicants to two programs we’re running in June: our 2016 Summer Fellows… Read more »
New paper on bounded Löb and robust cooperation of bounded agents
MIRI Research Fellow Andrew Critch has written a new paper on cooperation between software agents in the Prisoner’s Dilemma, available on arXiv: “Parametric bounded Löb’s theorem and robust cooperation of bounded agents.” The abstract reads: Löb’s theorem and Gödel’s theorem make predictions about the behavior of systems capable of self-reference with unbounded computational resources with… Read more »
