Updates A new paper from MIRI researcher Vanessa Kosoy, presented at the ICLR SafeML workshop this week: "Delegative Reinforcement Learning: Learning to Avoid Traps with a Little Help." New research posts: Learning "Known" Information When the Information is Not Actually Known; Defeating Goodhart and the "Closest Unblocked Strategy" Problem; Reinforcement Learning with Imperceptible Rewards The Long-Term Future Fund has announced twenty-three new… Read more »
Posts By: Rob Bensinger
New paper: “Delegative reinforcement learning”
MIRI Research Associate Vanessa Kosoy has written a new paper, “Delegative reinforcement learning: Learning to avoid traps with a little help.” Kosoy will be presenting the paper at the ICLR 2019 SafeML workshop in two weeks. The abstract reads: Most known regret bounds for reinforcement learning are either episodic or assume an environment without traps…. Read more »
April 2019 Newsletter
Updates New research posts: Simplified Preferences Needed, Simplified Preferences Sufficient; Smoothmin and Personal Identity; Example Population Ethics: Ordered Discounted Utility; A Theory of Human Values; A Concrete Proposal for Adversarial IDA MIRI has received a set of new grants from the Open Philanthropy Project and the Berkeley Existential Risk Initiative. News and links From the DeepMind safety team and Alex Turner: Designing Agent Incentives… Read more »
New grants from the Open Philanthropy Project and BERI
I’m happy to announce that MIRI has received two major new grants: A two-year grant totaling $2,112,500 from the Open Philanthropy Project. A $600,000 grant from the Berkeley Existential Risk Initiative. The Open Philanthropy Project’s grant was awarded as part of the first round of grants recommended by their new committee for effective altruism support:… Read more »
March 2019 Newsletter
Want to be in the reference class “people who solve the AI alignment problem”? We now have a guide on how to get started, based on our experience of what tends to make research groups successful. (Also on the AI Alignment Forum.) Other updates Demski and Garrabrant’s introduction to MIRI’s agent foundations research, “Embedded Agency,” is… Read more »
A new field guide for MIRIx
We’ve just released a field guide for MIRIx groups, and for other people who want to get involved in AI alignment research. MIRIx is a program where MIRI helps cover basic expenses for outside groups that want to work on open problems in AI safety. You can start your own group or find information on… Read more »
February 2019 Newsletter
Updates Ramana Kumar and Scott Garrabrant argue that the AGI safety community should begin prioritizing “approaches that work well in the absence of human models”: [T]o the extent that human modelling is a good idea, it is important to do it very well; to the extent that it is a bad idea, it is best… Read more »
January 2019 Newsletter
Our December fundraiser was a success, with 348 donors contributing just over $950,000. Supporters leveraged a variety of matching opportunities, including employer matching programs, WeTrust Spring’s Ethereum-matching campaign, Facebook’s Giving Tuesday event, and professional poker players Dan Smith, Aaron Merchak, and Martin Crowley’s Double Up Drive, expertly facilitated by Raising for Effective Giving. In all, matching… Read more »
