Interview with New MIRI Research Fellow Luke Muehlhauser

 |   |  Conversations

Section One: Background and Core Ideas

Q1. What is your personal background?
Q2. Why should we care about artificial intelligence?
Q3. Why do you think smarter-than-human artificial intelligence is possible?
Q4. The mission of the Machine Intelligence Research Institute is to “to ensure that the creation of smarter-than- human intelligence benefits society.” How is your research contributing to that mission?
Q5. How does MIRI’s approach to making friendly AI differ from the concept of Asimov laws?
Q6. Why is it necessary to make an AI that “€œwants the same things we want”?
Q7. If dangerous AI were to develop, why couldn’€™t we just “€œpull the plug”€?
Q8. Why are you and the Machine Intelligence Research Institute focused on artificial intelligence instead of human intelligence enhancement or whole brain emulation?


Read more »

2011 Summer Matching Challenge Success!

 |   |  News

Thanks to the effort of our donors, the 2011 Summer Singularity Challenge has been met! All $125,000 contributed will be matched dollar for dollar by our matching backers, raising a total of $250,000 to fund the Machine Intelligence Research Institute’€™s operations. We reached our goal two days early, near midnight of August 29th.

On behalf of our staff, volunteers, and entire community, I want to personally thank everyone who donated. Your dollars make the difference.

Here’€™s to a better future for the human species.

Machine Intelligence Research Institute Strategic Plan 2011

 |   |  News

The Machine Intelligence Research Institute has prepared a Strategic Plan (PDF) to concisely describe our near-term goals, vision, and concrete actions in pursuit of those goals and vision. We welcome commentary from our supporter community on the plan. The release of this Strategic Plan is part of an overall effort to increase transparency at the Institute.

Update: This strategic plan is now obsolete. See our 2013 strategic priorities instead.

Announcing the $125,000 Summer Singularity Challenge

 |   |  News

Thanks to the generosity of several major donors, every donation to the Machine Intelligence Research Institute made now until August 31, 2011 will be matched dollar-for-dollar, up to a total of $125,000.

Donate now!
Now is your chance to double your impact while supporting the Machine Intelligence Research Institute and helping us raise up to $250,000 to help fund our research program and stage the upcoming Singularity Summit… for which you can register now!

$125,000 in backing for this challenge is being generously provided by Rob Zahra, Quixey, Clippy, Luke Nosek, Edwin Evans, Rick Schwall, Brian Cartmell, Mike Blume, Jeff Bone, Johan Edsträm, Zvi Mowshowitz, John Salvatier, Louie Helm, Kevin Fischer, Emil Gilliam, Rob and Oksana Brazell, Guy Srinivasan, John Chisholm, and John Ku.

The year 2011 has been huge for Artificial Intelligence. With the IBM computer Watson defeating two top Jeopardy! champions in February, it’s clear that the field is making steady progress. Journalists like Torie Bosch of Slate have argued that “We need to move from robot-apocalypse jokes to serious discussions about the emerging technology.” We couldn’t agree more — in fact, the Machine Intelligence Research Institute has been thinking about how to create safe and ethical artificial intelligence since long before the Singularity landed on the front cover of TIME magazine.

The last 1.5 years were our biggest ever. Since the beginning of 2010, we have:

  • Held our annual Singularity Summit, in San Francisco. Speakers included Ray Kurzweil, James Randi, Irene Pepperberg, and many others.
  • Held the first Singularity Summit Australia and Singularity Summit Salt Lake City.
  • Held a wildly successful Rationality Minicamp.
  • Published seven research papers, including Yudkowsky’s much-awaited ‘Timeless Decision Theory‘.
  • Helped philosopher David Chalmers write his seminal paper ‘The Singularity: A Philosophical Analysis’, which has sparked broad discussion in academia, including an entire issue of Journal of Consciousness Studies and a book from Springer devoted to responses to Chalmers’ paper.
  • Launched the Research Associates program.
  • Brought MIT cosmologist Max Tegmark onto our advisory board, published our Singularity FAQ, and much more.

In the coming year, we plan to do the following:

  • Hold our annual Singularity Summit, in New York City this year.
  • Publish three chapters in the upcoming academic volume The Singularity Hypothesis, along with several other papers.
  • Improve organizational transparency by creating a simpler, easier-to-use website that includes Machine Intelligence Research Institute planning and policy documents.
  • Publish a document of open research problems related to Friendly AI, to clarify the research space and encourage other researchers to contribute to our mission.
  • Add additional skilled researchers to our Research Associates program.
  • Publish well-researched documents making the case for existential risk reduction as optimal philanthropy.
  • Diversify our funding sources by applying for targeted grants and advertising our affinity credit card program.

We appreciate your support for our high-impact work. As PayPal co-founder and Machine Intelligence Research Institute donor Peter Thiel said:

“I’m interested in facilitating a forum in which there can be… substantive research on how to bring about a world in which AI will be friendly to humans rather than hostile… [The Machine Intelligence Research Institute represents] a combination of very talented people with the right problem space [they’re] going after… [They’ve] done a phenomenal job… on a shoestring budget. From my perspective, the key question is always: What’s the amount of leverage you get as an investor? Where can a small amount make a big difference? This is a very leveraged kind of philanthropy.”

Donate now, and seize a better than usual chance to move our work forward. Credit card transactions are securely processed through, Google Checkout, or PayPal. If you have questions about donating, please call Amy Willey at (586) 381-1801.

Tallinn-Evans Challenge Grant Success!

 |   |  News

Thanks to the effort of our donors, the Tallinn-Evans Singularity Challenge has been met! All $125,000 contributed will be matched dollar for dollar by Jaan Tallinn and Edwin Evans, raising a total of $250,000 to fund the Machine Intelligence Research Institute’s operations in 2011. On behalf of our staff, volunteers, and entire community, I want to personally thank everyone who donated. Keep watching this blog throughout the year for updates on our activity, and sign up for our mailing list if you haven’t yet.

Here’s to a better future for the human species.

We are preparing a donor page to provide a place for everyone who donated to share some information about themselves if they wish, including their name, location, and a quote about why they donate to the Machine Intelligence Research Institute. If you would like to be included in our public list, please email us.

Again, thank you. The Machine Intelligence Research Institute depends entirely on contributions from individual donors to exist. Money is indeed the unit of caring, and one of the easiest ways that anyone can contribute directly to the success of the Machine Intelligence Research Institute. Another important way you can help is by plugging us into your networks, so please email us if you want to help.

If you’re interested in connecting with other Machine Intelligence Research Institute supporters, we encourage joining our group on Facebook. There are also local Less Wrong meetups in cities like San Francisco, Los Angeles, New York, and London.

Announcing the Tallinn-Evans $125,000 Singularity Challenge

 |   |  News

Thanks to the generosity of two major donors; Jaan Tallinn, a founder of Skype and Ambient Sound Investments, and Edwin Evans, CEO of the mobile applications startup Quinly, every contribution to the Machine Intelligence Research Institute up until January 20, 2011 will be matched dollar-for-dollar, up to a total of $125,000.






Interested in optimal philanthropy — that is, maximizing the future expected benefit to humanity per charitable dollar spent? The technological creation of greater-than-human intelligence has the potential to unleash an “intelligence explosion” as intelligent systems design still more sophisticated successors. This dynamic could transform our world as greatly as the advent of human intelligence has already transformed the Earth, for better or for worse. Thinking rationally about these prospects and working to encourage a favorable outcome offers an extraordinary chance to make a difference. The Machine Intelligence Research Institute exists to do so through its research, the Singularity Summit, and public education.

We support both direct engagements with the issues as well as the improvements in methodology and rationality needed to make better progress. Through our Visiting Fellows program, researchers from undergrads to Ph.Ds pursue questions on the foundations of Artificial Intelligence and related topics in two-to-three month stints. Our Resident Faculty, up to four researchers from three last year, pursues long-term projects, including AI research, a literature review, and a book on rationality, the first draft of which was just completed. MIRI researchers and representatives gave over a dozen presentations at half a dozen conferences in 2010. Our Singularity Summit conference in San Francisco was a great success, bringing together over 600 attendees and 22 top scientists and other speakers to explore cutting-edge issues in technology and science.

We are pleased to receive donation matching support this year from Edwin Evans of the United States, a long-time MIRI donor, and Jaan Tallinn of Estonia, a more recent donor and supporter. Jaan recently gave a talk on the Singularity and his life at a entrepreneurial group in Finland. Here’s what Jaan has to say about us:


“We became the dominant species on this planet by being the most intelligent species around. This century we are going to cede that crown to machines. After we do that, it will be them steering history rather than us. Since we have only one shot at getting the transition right, the importance of MIRI’s work cannot be overestimated. Not finding any organisation to take up this challenge as seriously as MIRI on my side of the planet, I conclude that it’s worth following them across 10 time zones.”
— Jaan Tallinn, MIRI donor

Make a lasting impact on the long-term future of humanity today — make a donation to the Machine Intelligence Research Institute and help us reach our $125,000 goal. For more detailed information on our projects and work, contact us at or read our new organizational overview.

2010 Singularity Research Challenge Fulfilled!

 |   |  News

Thanks to our donors, yesterday we met our fundraising goal of $100,000 for the 2010 Singularity Research Challenge. MIRI would like to thank the grant’s matching donors and everyone who contributed. Every donation, however small, funds research and advocacy targeted towards maximizing the probability of a positive Singularity.

If you have any questions or comments about MIRI’s activity or would like to discuss targeted donations for future projects, please feel free to contact us anytime at admin at intelligence dot org. We also encourage you to subscribe to this blog, if you haven’t already, to stay up-to-date on MIRI’s activity.

Again, thank you, and here’s to a productive and successful 2010!