This last year has been pretty astounding. Since its release twelve months ago, Nick Bostrom’s book Superintelligence has raised awareness about the challenge that MIRI exists to address: long-term risks posed by smarter-than-human artificially intelligent systems. Academic and industry leaders echoed these concerns in an open letter advocating “research aimed at ensuring that increasingly capable AI systems are robust and beneficial.” To jump-start this new safety-focused paradigm in AI, the Future of Life Institute has begun distributing $10M as grants to dozens of research groups, Bostrom and MIRI among them.
MIRI comes to this budding conversation with a host of relevant open problems already in hand. Indeed, a significant portion of the research priorities document accompanying the open letter is drawn from our work on this topic. Having already investigated these issues at some length, MIRI is well-positioned to shape this field as it enters a new phase in its development.
This is a big opportunity. MIRI is already growing and scaling its research activities, but the speed at which we scale in the coming months and years can be increased by more funding. For that reason, MIRI is starting a six-week fundraiser aimed at increasing our rate of growth.
And here it is!
— Progress Bar —
Rather than running a matching fundraiser with a single fixed donation target, we’ll be letting you help choose MIRI’s course, based on the details of our funding situation and how we would make use of marginal dollars. In particular, we’ll be blogging over the coming weeks about how our plans would scale up at different funding levels:
Target 1 — $250k: Continued growth. At this level, we would have enough funds to maintain a twelve-month runway while continuing all current operations, including running workshops, writing papers, and attending conferences. We will also be able to scale the research team up by one to three additional researchers, on top of our three current researchers and two new researchers who are starting this summer. This would ensure that we have the funding to hire the most promising researchers who come out of the MIRI Summer Fellows Program and our summer workshop series.
Target 2 — $500k: Accelerated growth. At this funding level, we could grow our team more aggressively, while maintaining a twelve-month runway. We would have the funds to expand the research team to about ten core researchers, while also taking on a number of exciting side-projects, such as hiring one or two type theorists. Recruiting specialists in type theory, a field at the intersection of computer science and mathematics, would enable us to develop tools and code that we think are important for studying verification and reflection in artificial reasoners.
Target 3 — $1.5M: Taking MIRI to the next level. At this funding level, we would start reaching beyond the small but dedicated community of mathematicians and computer scientists who are already interested in MIRI’s work. We’d hire a research steward to spend significant time recruiting top mathematicians from around the world, we’d make our job offerings more competitive, and we’d focus on hiring highly qualified specialists in relevant areas of mathematics. This would allow us to grow the research team as fast as is sustainable, while maintaining a twelve-month runway.
Target 4 — $3M: Bolstering our fundamentals. At this level of funding, we’d start shoring up our basic operations. We’d spend resources and experiment to figure out how to build the most effective research team we can. We’d branch out into additional high-value projects outside the scope of our core research program, such as hosting specialized conferences and retreats, upgrading our equipment and online resources, and running programming tournaments to spread interest about certain open problems. At this level of funding we’d also start extending our runway, and prepare for sustained aggressive growth over the coming years.
Target 5 — $6M: A new MIRI. At this point, MIRI would become a qualitatively different organization. With this level of funding, we would start forking the research team into multiple groups attacking the AI alignment problem from very different angles. Our current technical agenda is not the only way to approach the challenges that lie ahead — indeed, there are a number of research teams that we would be thrilled to start up inside MIRI given the opportunity.
We also have plans that extend beyond the $6M level: for more information, shoot me an email at contact@intelligence.org. I also invite you to email me with general questions or to set up a time to chat.
If you intend to make use of corporate matching (check here to see whether your employer will match your donation), email malo@intelligence.org and we’ll include the matching contributions in the fundraiser total.
Some of these targets are quite ambitious, and I’m excited to see what happens when we lay out the available possibilities and let our donors collectively decide how quickly we develop as an organization.
We’ll be using this fundraiser as an opportunity to explain our research and our plans for the future. If you have any questions about what MIRI does and why, email them to rob@intelligence.org. Answers will be posted to this blog every Monday and Friday.
Below is a list of explanatory posts written for this fundraiser, which we’ll be updating regularly:
July 1 — Grants and Fundraisers. Why we’ve decided to experiment with a multi-target fundraiser.
July 16 — An Astounding Year. Recent successes for MIRI, and for the larger field of AI safety.
July 18 — Targets 1 and 2: Growing MIRI. MIRI’s plans if we hit the $250k or $500k funding target.
July 20 — Why Now Matters. Two reasons to give now, rather than wait to give later.
July 24 — Four Background Claims. Basic assumptions behind MIRI’s focus on smarter-than-human AI.
July 27 — MIRI’s Approach. How we identify technical problems to work on.
July 31 — MIRI FAQ. Summarizing common sources of misunderstanding.
August 3 — When AI Accelerates AI. Some reasons to get started on safety work early.
August 7 — Target 3: Taking It To The Next Level. Our plans if we hit the $1.5M funding target.
August 10 — Assessing Our Past And Potential Impact. Why expect MIRI in particular to make a difference?
August 14 — What Sets MIRI Apart? Distinguishing MIRI from groups in academia and industry.
August 18 — Powerful Planners, Not Sentient Software. Why advanced AI isn’t “evil robots.”
August 28 — AI and Effective Altruism. On MIRI’s role in the EA community.
Our hope is that these new resources will help you, our donors, make more informed decisions during our fundraiser, and also that our fundraiser will serve as an opportunity for people to learn a lot more about our activities and strategic outlook.
As scientists, engineers, and policymakers begin to take notice of the AI alignment problem, MIRI is in a unique position to direct this energy and attention in a useful direction. Donating today will help us rise to this challenge and secure a place at the forefront of this critical field.
