Updates A new paper: “Forecasting Using Incomplete Models“ New research write-ups and discussions: Prisoners’ Dilemma with Costs to Modeling; Counterfactual Mugging Poker Game; Optimization Amplifies Eliezer Yudkowsky, Paul Christiano, Jessica Taylor, and Wei Dai discuss Alex Zhu’s FAQ for Paul’s research agenda. We attended EA Global in SF, and gave a short talk on “Categorizing Variants… Read more »
Posts By: Rob Bensinger
New paper: “Forecasting using incomplete models”
MIRI Research Associate Vadim Kosoy has a paper out on issues in naturalized induction: “Forecasting using incomplete models”. Abstract: We consider the task of forecasting an infinite sequence of future observations based on some number of past observations, where the probability measure generating the observations is “suspected” to satisfy one or more of a set… Read more »
June 2018 Newsletter
Updates New research write-ups and discussions: Logical Inductors Converge to Correlated Equilibria (Kinda) MIRI researcher Tsvi Benson-Tilsen and Alex Zhu ran an AI safety retreat for MIT students and alumni. Andrew Critch discusses what kind of advice to give to junior AI-x-risk-concerned researchers, and I clarify two points about MIRI’s strategic view. From Eliezer Yudkowsky: Challenges to… Read more »
May 2018 Newsletter
Updates New research write-ups and discussions: Resource-Limited Reflective Oracles; Computing An Exact Quantilal Policy New at AI Impacts: Promising Research Projects MIRI research fellow Scott Garrabrant and associates Stuart Armstrong and Vadim Kosoy are among the winners in the second round of the AI Alignment Prize. First place goes to Tom Everitt and Marcus Hutter’s “The Alignment Problem… Read more »
April 2018 Newsletter
Updates A new paper: “Categorizing Variants of Goodhart’s Law” New research write-ups and discussions: Distributed Cooperation; Quantilal Control for Finite Markov Decision Processes New at AI Impacts: Transmitting Fibers in the Brain: Total Length and Distribution of Lengths Scott Garrabrant, the research lead for MIRI’s agent foundations program, outlines focus areas and 2018 predictions for MIRI’s research. Scott presented on logical… Read more »
2018 research plans and predictions
Update Nov. 23: This post was edited to reflect Scott’s terminology change from “naturalized world-models” to “embedded world-models.” For a full introduction to these four research problems, see Scott Garrabrant and Abram Demski’s “Embedded Agency.” Scott Garrabrant is taking over Nate Soares’ job of making predictions about how much progress we’ll make in different research… Read more »
March 2018 Newsletter
Updates New research write-ups and discussions: Knowledge is Freedom; Stable Pointers to Value II: Environmental Goals; Toward a New Technical Explanation of Technical Explanation; Robustness to Scale New at AI Impacts: Likelihood of Discontinuous Progress Around the Development of AGI The transcript is up for Sam Harris and Eliezer Yudkowsky’s podcast conversation. Andrew Critch, previously on leave… Read more »
Sam Harris and Eliezer Yudkowsky on “AI: Racing Toward the Brink”
MIRI senior researcher Eliezer Yudkowsky was recently invited to be a guest on Sam Harris’ “Waking Up” podcast. Sam is a neuroscientist and popular author who writes on topics related to philosophy, religion, and public discourse. The following is a complete transcript of Sam and Eliezer’s conversation, AI: Racing Toward the Brink. Contents 1. Intelligence… Read more »
