Take Action

Many of the leading research scientists and research organizations in AI have warned that future, smarter-than-human AI systems pose a serious risk of causing human extinction.

The obvious response is to not build such systems. We propose that the international community act immediately to install an off-switch — the international agreements, policies, and physical infrastructure required to shut down hazardous AI projects before it’s too late. To this end, we are working to help policymakers get up to speed on the nature and the urgency of the problem.

But policymakers are not the only people with power. Leaders are responsive to the views of their constituents. The more the general public is demanding sane AI policy, the more likely policymakers are to focus on it.

You can help us spread the word.

Outreach and Resources

People with a wide variety of skills and backgrounds can help inform policymakers and the general public about extinction risk from future AI systems. There are a lot of options, from talking to your co-workers to hosting high-quality conversations on a podcast.

Awareness-raising can’t save the world on its own, but it currently seems like an important precursor to a robust policy response.

For many readers, then, this is our primary call to action: Start conversations with others about these risks, and ratchet up your understanding and the understanding of people around you. Here are some tips about how to do this effectively:

Don’t assume that others have it handled, until you can see with your own eyes that this is true. (Avoid the bystander effect.)
There’s room for a great deal of uncertainty about future AI development. But don’t confuse “the situation is uncertain” with “the best possible course of action is to do nothing”.  
When it comes to trying to predict the future trajectory of the field, experts are to a large extent flying blind. But this isn’t an excuse to race ahead on developing smarter-than-human AI, since most possible outcomes of deploying this technology would be extremely bad. 
If an engineer is highly uncertain about whether their bridge will collapse, they should not run vehicles over the bridge. It seems absurd to apply a far laxer standard to an invention the scientific community sees as posing a serious extinction risk for humanity; the case for action here is incredibly straightforward.
Finally, let your voice be heard. The development of artificial superintelligence will affect everyone, and everyone has a right to weigh in.

AI developers face complicated near-term incentives when it comes to regulating their own industry, and the key stakeholders in developing artificial superintelligence are the entire human race, not just the engineers building the system. While experts have a critical role to play in raising the alarm, we don’t believe that it takes years of technical experience to have an accurate understanding of why it would be a bad idea to rush into building vastly smarter-than-human AI systems.

Additionally, policymakers are likely to be much more responsive to the risks if subject matter experts and the general public are expressing serious concerns. The goal should be to foster a large number of conversations about this topic, occurring in parallel, at all levels of technical understanding.

What are some resources I can use for learning
more or sharing with others?

The main introductory resource we recommend is our Problem overview, which is also available as a two-page executive summary.

Some other explainers we’ve found useful include:

Short introductions from MIRI: our TED talk (11m watch) and op-ed in TIME (11m read).
Longer introductions from MIRI: a video interview with Bloomberg (24m watch), aimed at a more general audience; and AGI Ruin (28m read), aimed at a more technical audience.
In the Financial Times: “We must slow down the race to God-like AI” (17m read). Some shorter articles include pieces by CNN and the New York Times.
On Medium: “Preventing Extinction from Superintelligence” (10m read).

What are some tips for communicating usefully about AI risk?

At a high level:

Prioritize communication that scales well. For example, a podcast that gets heard by hundreds of people may be a better use of time than a private one-on-one conversation.
Keep in mind that building a large coalition is likely to be more useful than building a small but passionate one. For politicians to effectively act on this issue, there will likely need to be broad bipartisan and international support. Partisan polarization and entrenchment would likely be a very bad thing, and the ways individuals talk about AI risk can help shape whether the larger conversation ends up productive and substantive, versus ending up as a partisan tug-of-war.
Be wary of the fallacy “Something must be done, and this is something, so this must be done.” Think through which actions are actually likely to be the most useful.
Don’t make things worse. Unethical or manipulative conduct is a lot more likely to backfire than to help your cause.
Prioritize candor, openness, and honesty in trying to communicate about catastrophic AI risk. Ask questions about what others believe, and prioritize truth-seeking and integrity over winning every local argument. Just because the stakes are high, and time is of the essence, doesn’t mean that deception is more useful than honesty for putting the world in a marginally better position to prevent disaster.

In short: Do the obvious things you’d do if you were having a respectful conversation with people who are new to a topic, people who disagree with you, etc. Make mistakes, learn things, and be bold. Do the things that are likely to actually work, not just the things that sound good on paper. The stakes are too high for anything less than that.

Careers and Donation Targets

MIRI is actively looking for more funders for our work; see our Donation page for details.

We’re also in the process of growing our communications team and our technical governance team. Our job applications are closed as of 2024-10-17, but we’ll be announcing when they re-open on our monthly mailing list and on X/Twitter.

There are a number of other organizations that we think are doing good work that’s relevant to AI risk. If you’re interested in doing full-time work on this issue, we recommend applying to openings at these organizations. We also recommend these groups as donation targets, and as models for the kinds of work we’d like to see more organizations and projects doing in the world.

AI Policy Institute — donate
AISafety.info — donate
Apollo Research — jobs
BlueDot Impact — jobs
Center for AI Safety — jobs, donate
Center for Applied Rationality — donate
Civic AI Security Program — jobs
Encode Justice — donate
Epoch AI — jobs, donate
Future of Life Institute — jobs
Lightcone Infrastructure — jobs, donate
Long-Term Future Fund — donate
Manifold Markets
Palisade Research — jobs
Rational Animations
Redwood Research — jobs
SeedAI
The Technology and Security Policy Center at RAND — jobs
UK AI Safety Institute — jobs

This isn’t an exhaustive list, and we don’t fully endorse or agree with all of these groups (or vice versa); but we think all of them are, in expectation, doing legitimately useful work.

More generally, some of the projects and careers we consider relatively promising for helping with catastrophic AI risk are:

Policy and advocacy — There are many useful policy initiatives to push on, but we expect work to be especially valuable insofar as it directly helps move the US and the international community toward installing an off-switch.
General awareness-raising — Improving the public’s general understanding of these issues is potentially critical for an effective policy response, but it’s also critical for the world’s overall ability to navigate future AI developments. Some future developments will be surprising, and some avenues for hope may come from unexpected places. It isn’t enough to just advocate for a specific policy that looks sensible today; decision-makers will need a better grasp of this issue if they’re to respond well in the future.
Human enhancement and whole-brain emulation — Researchers to date have been relatively unproductive at solving technical alignment problems. Qualitatively improving or speeding up the world’s top talent could help solve that problem, making it possible for alignment to “catch up” with AI capabilities. This seems like a longshot, but worth pursuing in parallel to regulatory solutions.
Computer and information security — Security skills and methodology have a lot of applications to AI alignment, but they may be especially valuable for improving information security at leading AI labs. This is a lower priority than shutting down development of dangerous technology, but it’s still a genuine positive to not have the dangerous tech leak onto the public Internet, or end up in the hands of determined and well-resourced attackers. Among other things, this can buy the world more time to put an off switch in place, and it can make off switches effective even after a model has been trained.
AI alignment — This research avenue seems more important in the long run than in the short run, since the odds of us succeeding on superintelligence alignment seem very low. Still, it seems worth pursuing in parallel to other useful work. Our main concern with supporting AI alignment research is that it can often overlap with capabilities work, making it easy for alignment work to cause more harm than good. Still, we expect the benefits to outweigh the costs in some cases; e.g., we’re strong supporters of the work of Chris Olah’s mechanistic interpretability team, and think they’ve put out some fascinating results.
Forecasting, rationality, and institution reform — Improving policymakers’ ability to coordinate and make decisions about emerging technologies seems potentially very leveraged, though we’re pessimistic about how much impact these efforts can have on short timescales.