How Big is the Field of Artificial Intelligence? (initial findings)

 |   |  Analysis

Co-authored with Jonah Sinick.

How big is the field of AI, and how big was it in the past?

This question is relevant to several issues in AGI safety strategy. To name just two examples:

  • AI forecasting. Some people forecast AI progress by looking at how much has been accomplished for each calendar year of research. But as inputs to AI progress, (1) AI funding, (2) quality-adjusted researcher years (QARYs), and (3) computing power are more relevant than calendar years.1 To use these metrics to predict future AI progress, we need to know how many dollars and QARYs and computing cycles at various times in the past have been required to produce the observed progress in AI thus far.
  • Leverage points. If most AI research funding comes from relatively few funders, or if most research is produced by relatively few research groups, then these may represent high-value leverage points through which one might influence the field as a whole, e.g. to be more concerned with the long-term social consequences of AI.

For these reasons and more, MIRI recently investigated the current size and past growth of the AI field. This blog post summarizes our initial findings, which are meant to provide a “quick and dirty” launchpad for future, more thorough research into the topic.

Read more »

  1. Another important input metric is theoretical progress imported from other fields, e.g. methods from statistics. ↩

Existential Risk Strategy Conversation with Holden Karnofsky

 |   |  Conversations, MIRI Strategy

On January 16th, 2014, MIRI met with Holden Karnofsky to discuss existential risk strategy. The participants were:

We recorded and transcribed the conversation, and then edited and paraphrased the transcript for clarity, conciseness, and to protect the privacy of some content. The resulting edited transcript is available in full here (41 pages).

Below is a summary of the conversation written by Karnofsky, then edited by Muehlhauser and Yudkowsky. Below the summary are some highlights from the conversation chosen by Karnofsky.

See also three previous conversations between MIRI and Holden Karnofsky: on MIRI strategy, on transparent research analyses, and on flow-through effects.

Read more »

2013 in Review: Outreach

 |   |  MIRI Strategy

This is the 2nd part of my personal and qualitative self-review of MIRI in 2013.

By “outreach” I refer to general outreach efforts, rather than e.g. outreach to specific researchers, which will be discussed in the post about MIRI’s 2013 research activities.

 

Outeach in 2013

  1. In early 2013, we decided to reduce our outreach efforts significantly, and we did.
  2. However, we learned throughout 2013 that some forms of “indirect” outreach tend to be pretty cost-effective for MIRI’s goals.1
  3. Therefore, we plan to put more effort into indirect outreach in 2014 than we did in 2013.

Read more »

  1. We are currently gathering much more information about the effects of our direct and indirect outreach efforts, but these data will take several months to gather. It’s possible those data will overturn the basic conclusions in this post, but it’s more likely they will slightly adjust them. ↩

Want to help MIRI by investing in XRP?

 |   |  News

XRP

Recently, Mt.Gox and Ripple creator Jed McCaleb gave MIRI a large donation in XRP, which is the #2 cryptocurrency in market cap behind Bitcoin. This gift, along with our recent successful fundraiser, should enable us to hire the beginnings of a full-time Friendly AI research team in 2014. It’s difficult to know exactly how to value the donation, but (e.g.) valuing it at the exchange rate at the time of donation would make it the largest single donation to MIRI in our history.

The size of Jed’s donation means that we currently have more XRP than it makes sense to hold in our diversified asset portfolio. To reduce our XRP holdings while growing the Ripple user base, we’d like to sell some of our XRP to members of our community. If you’re interested, please contact Malo Bourgon (malo@intelligence.org).

How does Ripple/XRP work? Technically, Ripple is an online payments protocol and XRP is an associated digital currency, but XRP is often referred to as “Ripple,” which can be confusing. The best explanation of all this yet written is “Bitcoin Vs. Ripple” from the Coinsetter blog, which I’ll excerpt below:

…think of Ripple as “Kayak for currency exchange.” Ripple will compare various pathways of exchanging one currency to another and find the lowest cost option. Transactions also happen rapidly (in under 5 seconds)…

XRP is a digital currency… with an embedded use, which is to pay for Ripple transactions, create Ripple accounts, and be a currency of last resort in the Ripple system… Whereas Bitcoin has no inherent use (its demand and use is solely based on people’s interest in Bitcoin), XRP has… a basic reason why someone may need to use XRP, which is to complete Ripple transactions.

…Should Bitcoin believers be worried that Ripple is on its way to crush their dream currency? Absolutely not. Interest in Bitcoin will continue to grow, and separately, interest in Ripple will continue to grow. In fact, the Ripple network should help make bitcoins substantially easier to buy and sell, as well as legitimize them.

Click here to create a Ripple wallet and join the network; the process takes less than 60 seconds.

MIRI continues to accept donations in both XRP and Bitcoin, and we send tax receipts for non-anonymous donations in either currency.

MIRI’s January 2014 Newsletter

 |   |  Newsletters

Machine Intelligence Research Institute

Dear friends,Wow! MIRI’s donors finished our Winter 2013 fundraising drive three weeks early. Our sincere thanks to everyone who contributed.

Research Updates

Other Updates

Other Links

As always, please don’t hesitate to let us know if you have any questions or comments.

Best,
Luke Muehlhauser
Executive Director

MIRI strategy conversation with Steinhardt, Karnofsky, and Amodei

 |   |  Conversations, MIRI Strategy

On October 27th, 2013, MIRI met with three additional members of the effective altruism community to discuss MIRI’s organizational strategy. The participants were:

We recorded and transcribed much of the conversation, and then edited and paraphrased the transcript for clarity, conciseness, and to protect the privacy of some content. The resulting edited transcript is available in full here (62 pages).

Our conversation located some disagreements between the participants; these disagreements are summarized below. This summary is not meant to present arguments with all their force, but rather to serve as a guide to the reader for locating more information about these disagreements. For each point, a page number has been provided for the approximate start of that topic of discussion in the transcript, along with a phrase that can be searched for in the text. In all cases, the participants would likely have quite a bit more to say on the topic if engaged in a discussion on that specific point.

Read more »

Kathleen Fisher on High-Assurance Systems

 |   |  Conversations

Kathleen Fisher portraitDr. Kathleen Fisher joined DARPA as a program manager in 2011. Her research and development interests relate to programming languages and high assurance systems. Dr. Fisher joined DARPA from Tufts University. Previously, she worked as a Principal Member of the technical staff at AT&T Labs. Dr. Fisher received her Doctor of Philosophy in computer science and her Bachelor of Science in math and computational science from Stanford University.

Luke Muehlhauser: Kathleen, you’re the program manager at DARPA for HACMS program, which aims to construct cyber-physical systems which satisfy “appropriate safety and security properties” using a “clean-slate, formal methods-based approach.” My first question is similar to one I asked Greg Morrisett about the SAFE program, which aims for a “clean slate design for resilient and secure systems”: In the case of HACMS, why was it so important to take the “clean slate” approach, and design the system from the ground up for safety and security (along with functional correctness)?

Kathleen Fisher: Researchers have been trying to prove programs correct for decades, with very little success until recently (successful examples include NICTA’s seL4 microKernel and INRIA’s compCert verifying C compiler). One of the lessons learned in this process is that verifying existing code is much harder than verifying code written with verification in mind.

There are a couple of reasons for this. First, many of the invariants that have to hold for a program to be correct often exist only in the head of the programmer. When trying to verify a program after the fact, these invariants have been lost and can take a very long time for the verifier to rediscover. Second, sometimes the code can be written in multiple ways, some of which are much easier to verify than others. If programmers know they have to verify the code, they’ll choose the way that is easy to verify.

Read more »

Donor Story #1: Noticing Inferential Distance

 |   |  Guest Posts

2013 was by far MIRI’s most successful fundraising year (more details later), so now we’re talking to our donors to figure out: “Okay, what exactly are we doing so right?”

Below is one donor’s story, anonymized and published with permission:

My decision to donate was heavily dependent upon MIRI’s relationship with LessWrong. I did look into MIRI itself, perused the blog and the papers, and did some fact-checking. But this was largely sanity-checking after I had been convinced to donate by my interactions on LessWrong.

Initially, I wasn’t so much convinced to donate as I was convinced that FAI is a more pressing problem than my prior concerns. Once I believed this, it wasn’t a question of whether I was going to donate to FAI research but a question of where to focus my efforts.

I chose to donate to MIRI after it became apparent to me that

  1. Few people are aware of the problems posed by uFAI.
  2. Among those who are, many dismiss the problem after failing to understand it.

This perhaps sounds arrogant. Allow me to explain some:

Read more »