Updates A new paper: “Categorizing Variants of Goodhart’s Law” New research write-ups and discussions: Distributed Cooperation; Quantilal Control for Finite Markov Decision Processes New at AI Impacts: Transmitting Fibers in the Brain: Total Length and Distribution of Lengths Scott Garrabrant, the research lead for MIRI’s agent foundations program, outlines focus areas and 2018 predictions for MIRI’s research. Scott presented on logical… Read more »
Posts By: Rob Bensinger
2018 research plans and predictions
Scott Garrabrant is taking over Nate Soares’ job of making predictions about how much progress we’ll make in different research areas this year. Scott divides MIRI’s alignment research into five categories: naturalized world-models — Problems related to modeling large, complex physical environments that lack a sharp agent/environment boundary. Central examples of problems in this category include… Read more »
March 2018 Newsletter
Updates New research write-ups and discussions: Knowledge is Freedom; Stable Pointers to Value II: Environmental Goals; Toward a New Technical Explanation of Technical Explanation; Robustness to Scale New at AI Impacts: Likelihood of Discontinuous Progress Around the Development of AGI The transcript is up for Sam Harris and Eliezer Yudkowsky’s podcast conversation. Andrew Critch, previously on leave… Read more »
Sam Harris and Eliezer Yudkowsky on “AI: Racing Toward the Brink”
MIRI senior researcher Eliezer Yudkowsky was recently invited to be a guest on Sam Harris’ “Waking Up” podcast. Sam is a neuroscientist and popular author who writes on topics related to philosophy, religion, and public discourse. The following is a complete transcript of Sam and Eliezer’s conversation, AI: Racing Toward the Brink. Contents 1. Intelligence… Read more »
February 2018 Newsletter
Updates New at IAFF: An Untrollable Mathematician New at AI Impacts: 2015 FLOPS Prices We presented “Incorrigibility in the CIRL Framework” at the AAAI/ACM Conference on AI, Ethics, and Society. From MIRI researcher Scott Garrabrant: Sources of Intuitions and Data on AGI News and links In “Adversarial Spheres,” Gilmer et al. investigate the tradeoff between test… Read more »
January 2018 Newsletter
Our 2017 fundraiser was a huge success, with 341 donors contributing a total of $2.5 million! Some of the largest donations came from Ethereum inventor Vitalik Buterin, bitcoin investors Christian Calderon and Marius van Voorden, poker players Dan Smith and Tom and Martin Crowley (as part of a matching challenge), and the Berkeley Existential Risk… Read more »
End-of-the-year matching challenge!
Update 2017-12-27: We’ve blown past our 3rd and final target, and reached the matching cap of $300,000 for the Matching Challenge! Thanks so much to everyone who supported us! All donations made before 23:59 PST on Dec 31st will continue to be counted towards our fundraiser total. The fundraiser total includes projected matching funds from… Read more »
December 2017 Newsletter
Our annual fundraiser is live. Discussed in the fundraiser post: News — What MIRI’s researchers have been working on lately, and more. Goals — We plan to grow our research team 2x in 2018–2019. If we raise $850k this month, we think we can do that without dipping below a 1.5-year runway. Actual goals — A bigger-picture outline of… Read more »
