This is the 4th part of my personal and qualitative self-review of MIRI in 2013, in which I review MIRI’s 2013 Friendly AI (FAI) research activities.¹

Friendly AI research in 2013

1. In early 2013, we decided to shift our priorities from research plus public outreach to a more exclusive focus on technical FAI research. This resulted in roughly as much public-facing FAI research in 2013 as in all past years combined.
2. Also, our workshops succeeded in identifying candidates for hire. We expect to hire two 2013 workshop participants in the first half of 2014.
3. During 2013, I learned many things about how to create an FAI research institute and FAI research field. In particular…
4. MIRI needs to attract more experienced workshop participants.
5. Much FAI research can be done by a broad community, and need not be labeled as FAI research. But, more FAI progress is made when the researchers themselves conceive of the research as FAI research.
6. Communication style matters a lot.

Read more »

1. What counts as “Friendly AI research” is, naturally, a matter of debate. For most of this post I’ll assume “Friendly AI research” means “what Yudkowsky thinks of as Friendly AI research,” with the exception of intelligence explosion microeconomics, for reasons given in this post. ↩

This is the 3rd part of my personal and qualitative self-review of MIRI in 2013, in which I begin to review MIRI’s 2013 research activities. By “research activities” I mean to include outreach efforts primarily aimed at researchers, and also three types of research performed by MIRI:

• Expository research aims to consolidate and clarify already-completed strategic research or Friendly AI research that hasn’t yet been explained with sufficient clarity or succinctness, e.g. “Intelligence Explosion: Evidence and Import” and “Robust Cooperation: A Case Study in Friendly AI Research.” (I consider this a form of “research” because it often requires significant research work to explain ideas clearly, cite relevant sources, etc.)
• Strategic research aims to clarify how the future is likely to unfold, and what we can do now to nudge the future toward good outcomes, and involves more novel thought and modeling than expository research — though, the distinction is fuzzy. See e.g. “Intelligence Explosion Microeconomics” and “How We’re Predicting AI — or Failing to.”¹
• Friendly AI research aims to solve the technical sub-problems that seem most relevant to the challenge of designing a stably self-improving artificial intelligence with humane values. This often involves sharpening philosophical problems into math problems, and then developing the math problems into engineering problems. See e.g. “Tiling Agents for Self-Modifying AI” and “Robust Cooperation in the Prisoner’s Dilemma.”

I’ll review MIRI’s strategic and expository research in this post; my review of MIRI’s 2013 Friendly AI research will appear in a future post. For the rest of this post, I usually won’t try to distinguish which writings are “expository” vs. “strategic” research, since most of them are partially of both kinds.

Read more »

1. Note that what I call “MIRI’s strategic research” or “superintelligence strategy research” is a superintelligence-focused subset of what GiveWell would call “strategic cause selection research” and CEA would call this “cause prioritization research.” ↩

In late 2011, MIRI opened a Google Grants account, which provides $10k/mo in free Google Adwords for nonprofits. Kevin Fischer and I tweaked our Adwords account over several months until we successfully spent the full $10,000/mo 3 months in a row.

This qualified us for Grants Pro, and we are still grandfathered into that (now unavailable) $40k/mo level of free Adwords. The limit is actually $1350/day which is challenging to spend wisely, even with the recently increased max bidding level of $2/click. But with more tweaking we are now able to spend nearly all of it each month. Kevin and I probably spent 100 hours between us over the past few years optimizing this, but much of it was done while we were both volunteering for MIRI. Ongoing tweaking requires only an hour or less of my time per month.

The traffic is large (2/3 of our total) but marginal in quality. In the past 6 months, we drove ~250,000 visitors to MIRI’s site via Google Adwords.

• 5000 people read at least one of our research papers
• 500 signed up for the newsletter (our true goal since it gives them a chance to hear from us again)
• 150 went to the volunteer site (and almost all didn’t sign up once they got there)
• 100 applied to attend research workshops (no qualified candidates yet)

Our impression is that MIRI has an especially hard time making good use of Google Adwords, because there is such a gulf of inferential distance between what we do and what people already know. Many things we could show someone who had never heard about us to try to have a strong impact would plausibly be more misleading than helpful. We expect most charities could make higher-value use of Google Adwords than we can for this reason, including e.g. effective altruism meta-charities or animal welfare groups.

Paying $16 for each newsletter subscriber is pretty bad, and it’s not remotely how we would spend $1350/day if it was unrestricted money. But, Adwords creates some value on the margin and we’re glad Google includes us in the program and that we’re able to reach new people about our work by using it. It’s 1000 new people getting our newsletter every year, and more eyeballs on our content. Some of them might pass it to someone else who would be good for a workshop, or something.

Also a word of warning to other nonprofits: we tried many times to get interns or volunteers to improve our Adwords account, but nobody was good at it. Lots of remote supporters (some of whom swore they were amazing at AdWords) were given access and didn’t make a single change to a single campaign, much less create new experiments and find improvements. We also briefly tried paying a contractor to make new ads and they at least tried things, but their ads didn’t generate value so we had to let them go.

If charities work with volunteers or supporters to improve their Adwords accounts, I’d recommend requiring that volunteers produce proof that they are currently managing at least one other large Adwords account successfully (or only ask volunteers for cheap things like ideas and don’t expect any help from them doing the much more costly and difficult work of actually implementing their ideas).