New paper: “Concept learning for safe autonomous AI”

 |   |  Papers

Concept learningMIRI research associate Kaj Sotala has released a new paper, accepted to the AI & Ethics workshop at AAAI-2015, titled “Concept learning for safe autonomous AI.”

The abstract reads:

Sophisticated autonomous AI may need to base its behavior on fuzzy concepts such as well-being or rights. These concepts cannot be given an explicit formal definition, but obtaining desired behavior still requires a way to instill the concepts in an AI system. To solve the problem, we review evidence suggesting that the human brain generates its concepts using a relatively limited set of rules and mechanisms. This suggests that it might be feasible to build AI systems that use similar criteria for generating their own concepts, and could thus learn similar concepts as humans do. Major challenges to this approach include the embodied nature of human thought, evolutionary vestiges in cognition, the social nature of concepts, and the need to compare conceptual representations between humans and AI systems.

December newsletter

 |   |  Newsletters

 

Machine Intelligence Research Institute

MIRI’s winter fundraising challenge has begun! Every donation made to MIRI between now and January 10th will be matched dollar-for-dollar, up to a total of $100,000!

 

Donate now to double your impact while helping us raise up to $200,000 (with matching) to fund our research program.

Research Updates

News Updates

Other Updates

  • Our friends at the Center for Effective Altruism will pay you $1,000 if you introduce them to somebody new that they end up hiring for one of their five open positions.

As always, please don’t hesitate to let us know if you have any questions or comments.

Best,
Luke Muehlhauser
Executive Director

 

2014 Winter Matching Challenge!

 |   |  News

Update: we have finished the matching challenge! Thanks everyone! The original post is below.

Nate & Nisan

Thanks to the generosity of Peter Thiel,1 every donation made to MIRI between now and January 10th will be matched dollar-for-dollar, up to a total of $100,000!

$0

$25K

$50K

$75K

$100K

We have reached our matching total of $100,000!

83

Total Donors

Now is your chance to double your impact while helping us raise up to $200,000 (with matching) to fund our research program.

Corporate matching and monthly giving pledges will count towards the total! Check here to see whether your employer will match your donation. Please email malo@intelligence.org if you intend to make use of corporate matching, or if you’d like to pledge 6 months of monthly donations, so that we can properly account for your contributions. If making use of corporate matching, make sure to donate before the end of the year so that you don’t unnecessarily “leave free money on the table” from your employer!

If you’re unfamiliar with our mission, see: Why MIRI?

Donate Now

 

Accomplishments Since Our Summer 2014 Fundraiser Launched:

Donate Now

Your Donations Will Support:

  • As mentioned above, we’re finishing up several more papers and technical reports, including an overview of our technical agenda so far.
  • We’re preparing the launch of an invite-only discussion forum devoted exclusively to technical FAI research. Beta users (who are also FAI researchers) have already posted more than a dozen technical discussions to the beta website. These will be available for all to see once the site launches publicly.
  • We continue to grow the MIRIx program, mostly to enlarge the pool of people we can plausibly hire as full-time FAI researchers in the next couple years.
  • We’re planning, or helping to plan, multiple research workshops, including the May 2015 decision theory workshop at Cambridge University.
  • We continue to host visiting researchers. For example in January we’re hosting Patrick LaVictoire and Matt Elder for multiple weeks.
  • We’re finishing up several more strategic analyses, on AI safety and on the challenges of preparing wisely for disruptive technological change in general.
  • We’re finishing the editing for a book version of Eliezer’s Sequences.
  • We’re helping to fund further SPARC programs, which provide education and skill-building to elite young math talent, and introduces them to ideas like effective altruism and global catastrophic risks.

Other projects are being surveyed for likely cost and impact. See also our mid-2014 strategic plan.

We appreciate your support for our work! Donate now, and seize a better than usual opportunity to move our work forward.

If you have questions about donating, please contact me (Luke Muehlhauser) at luke@intelligence.org.2

  1. Peter Thiel has pledged $150,000 to MIRI unconditionally, and an additional $100,000 conditional on us being able to raise matched funds from other donors. Hence this year our winter matching challenge goal is $100,000. Another reason this year’s winter fundraiser is smaller than last year’s winter challenge is that we’ve done substantially more fundraising before December this year than we did before December last year. ↩
  2. In particular, we expect that many of our donors holding views aligned with key ideas of effective altruism may want to know not just that donating to MIRI now will do some good but that donating to MIRI now will plausibly do more good than donating elsewhere would do (on the present margin, given the individual donor’s altruistic priorities and their model of the world). Detailed comparisons are beyond the scope of this announcement, but I have set aside time in my schedule to take phone calls with donors who would like to discuss such issues in detail, and I encourage you to email me to schedule such a call if you’d like to. (Also, I don’t have many natural opportunities to chat with most MIRI donors anyway, and I’d like to be doing more of it, so please don’t hesitate to email me and schedule a call!)  ↩

Three misconceptions in Edge.org’s conversation on “The Myth of AI”

 |   |  Analysis

A recent Edge.org conversation — “The Myth of AI” — is framed in part as a discussion of points raised in Bostrom’s Superintelligence, and as a response to much-repeated comments by Elon Musk and Stephen Hawking that seem to have been heavily informed by Superintelligence.

Unfortunately, some of the participants fall prey to common misconceptions about the standard case for AI as an existential risk, and they probably haven’t had time to read Superintelligence yet.

Of course, some of the participants may be responding to arguments they’ve heard from others, even if they’re not part of the arguments typically made by FHI and MIRI. Still, for simplicity I’ll reply from the perspective of the typical arguments made by FHI and MIRI.1

 

1. We don’t think AI progress is “exponential,” nor that human-level AI is likely ~20 years away.

Lee Smolin writes:

I am puzzled by the arguments put forward by those who say we should worry about a coming AI, singularity, because all they seem to offer is a prediction based on Moore’s law.

That’s not the argument made by FHI, MIRI, or Superintelligence.

Some IT hardware and software domains have shown exponential progress, and some have not. Likewise, some AI subdomains have shown rapid progress of late, and some have not. And unlike computer chess, most AI subdomains don’t lend themselves to easy measures of progress, so for most AI subdomains we don’t even have meaningful subdomain-wide performance data through which one might draw an exponential curve (or some other curve).

Thus, our confidence intervals for the arrival of human-equivalent AI tend to be very wide, and the arguments we make for our AI timelines are fox-ish (in Tetlock’s sense). Read more »

  1. I could have also objected to claims and arguments made in the conversation, for example Lanier’s claim that “The AI component would be only ambiguously there and of little importance [relative to the actuators component].” To me, this is like saying that humans rule the planet because of our actuators, not because of our superior intelligence. Or in response to Kevin Kelly’s claim that “So far as I can tell, AIs have not yet made a decision that its human creators have regretted,” I can for example point to the automated trading algorithms that nearly bankrupted Knight Capital faster than any human could react. But in this piece I will focus instead on claims that seem to be misunderstandings of the positive case that’s being made for AI as an existential risk. ↩

A new guide to MIRI’s research

 |   |  News

Guide to MIRI's ResearchNate Soares has written “A Guide to MIRI’s Research,” which outlines the main thrusts of MIRI’s current research agenda and provides recommendations for which textbooks and papers to study so as to understand what’s happening at the cutting edge.

This guide replaces Louie Helm’s earlier “Recommended Courses for MIRI Math Researchers,” and will be updated regularly as new lines of research open up, and as new papers and reports are released. It is not a replacement for our upcoming technical report on MIRI’s current research agenda and its supporting papers, which are still in progress. (“Corrigibility” is the first supporting paper we’ve released for that project.)

MIRI’s November Newsletter

 |   |  Newsletters

 

 

Machine Intelligence Research Institute

Research Updates

News Updates

Other Updates

  • Elon Musk has been talking about AGI risk quite a bit lately. He has also joined the external advisory boards of CSER and FLI, two existential risk organizations with whom we work closely.
  • Effective altruists may want to register for the EA Donation Registry.

As always, please don’t hesitate to let us know if you have any questions or comments.

Best,
Luke Muehlhauser
Executive Director

 

The Financial Times story on MIRI

 |   |  Analysis

Richard Waters wrote a story on MIRI and others for Financial Times, which also put Nick Bostrom’s Superintelligence at the top of its summer science reading list.

It’s a good piece. Go read it and then come back here so I can make a few clarifications.

 

1. Smarter-than-human AI probably isn’t coming “soon.”

“Computers will soon become more intelligent than us,” the story begins, but few experts I know think this is likely.

A recent survey asked the world’s top-cited living AI scientists by what year they’d assign a 10% / 50% / 90% chance of human-level AI (aka AGI), assuming scientific progress isn’t massively disrupted. The median reply for a 10% chance of AGI was 2024, for a 50% chance of AGI it was 2050, and for a 90% chance of AGI it was 2070. So while AI scientists think it’s possible we might get AGI soon, they largely expect AGI to be an issue for the second half of this century.

Moreover, many of those who specialize in thinking about AGI safety actually think AGI is further away than the top-cited AI scientists do. For example, relative to the surveyed AI scientists, Nick Bostrom and I both think more probability should be placed on later years. We advocate more work on the AGI safety challenge today not because we think AGI is likely in the next decade or two, but because AGI safety looks to be an extremely difficult challenge — more challenging than managing climate change, for example — and one requiring several decades of careful preparation.

The greatest risks from both climate change and AI are several decades away, but thousands of smart researchers and policy-makers are already working to understand and mitigate climate change, and only a handful are working on the safety challenges of advanced AI. On the present margin, we should have much less top-flight cognitive talent going into climate change mitigation, and much more going into AGI safety research.

Read more »