MIRI’s April newsletter: Relaunch Celebration and a New Math Result
|
|||
|
Dear friends,MIRI’s Strategy for 2013
This post is not a detailed strategic plan. For now, I just want to provide an update on what MIRI is doing in 2013 and why.
Our mission remains the same. The creation of smarter-than-human intelligence will likely be the most significant event in human history, and MIRI exists to help ensure that this event has a positive impact.
Still, much has changed in the past year:
- The short-term goals in our August 2011 strategic plan were largely accomplished.
- We changed our name from “The Singularity Institute” to “The Machine Intelligence Research Institute” (MIRI).
- We were once doing three things — research, rationality training, and the Singularity Summit. Now we’re doing one thing: research. Rationality training was spun out to a separate organization, CFAR, and the Summit was acquired by Singularity University. We still co-produce the Singularity Summit with Singularity University, but this requires limited effort on our part.
- After dozens of hours of strategic planning in January–March 2013, and with input from 20+ external advisors, we’ve decided to (1) put less effort into public outreach, and to (2) shift our research priorities to Friendly AI math research.
It’s this last pair of changes I’d like to explain in more detail below.
Facing the Intelligence Explosion ebook
Facing the Intelligence Explosion is now available as an ebook!
You can get it here. It is available as a “pay-what-you-want” package that includes the ebook in three formats: MOBI, EPUB, and PDF.
It is also available on Amazon Kindle (US, Canada, UK, and most others) and the Apple iBookstore (US, Canada, UK and most others).
All sources are DRM-free. Grab a copy, share it with your friends, and review it on Amazon or the iBookstore.
All proceeds go directly to funding the technical and strategic research of the Machine Intelligence Research Institute.
The Lean Nonprofit
Can Lean Startup methods work for nonprofits?
The Lean Startup‘s author Eric Ries seems to think so:
A startup is a human institution designed to create a new product or service under conditions of extreme uncertainty… Anyone who is creating a new product or business under conditions of extreme uncertainty is an entrepreneur whether he or she knows it or not, and whether working in a government agency, a venture-backed company, a nonprofit, or a decidedly for-profit company with financial investors.
In the past year, I helped launch one new nonprofit (Center for Applied Rationality), I massively overhauled one older nonprofit (MIRI), and I consulted with many nonprofit CEOs and directors. Now I’d like to share some initial thoughts on the idea of a “Lean Nonprofit.”
Early draft of naturalistic reflection paper
Update: See Reflection in Probabilistic Logic for more details on how this result relates to MIRI’s research mission.
In a recent blog post we described one of the results of our 1st MIRI Workshop on Logic, Probability, and Reflection:
The participants worked on the foundations of probabilistic reflective reasoning. In particular, they showed that a careful formalization of probabilistic logic can circumvent many classical paradoxes of self-reference. Applied to metamathematics, this framework provides (what seems to be) the first definition of truth which is expressive enough for use in reflective reasoning.
In short, the result described is a “loophole” in Tarski’s undefinability theorem (1936).
An early draft of the paper describing this result is now available: download it here. Its authors are Paul Christiano (UC Berkeley), Eliezer Yudkowsky (MIRI), Marcello Herreshoff (Google), and Mihály Bárász (Google). An excerpt from the paper is included below:
Unfortunately, it is impossible for any expressive language to contain its own truth predicate True…
There are a few standard responses to this challenge.
The first and most popular is to work with meta-languages…
A second approach is to accept that some sentences, such as the liar sentence G, are neither true nor false…
Although this construction successfully dodges the “undefinability of truth” it is somewhat unsatisfying. There is no predicate in these languages to test if a sentence… is undefined, and there is no bound on the number of sentences which remain undefined. In fact, if we are specifically concerned with self-reference, then a great number of properties of interest (and not just pathological counterexamples) become undefined.
In this paper we show that it is possible to perform a similar construction over probabilistic logic. Though a language cannot contain its own truth predicate True, it can nevertheless contain its own “subjective probability” function P. The assigned probabilities can be reflectively consistent in the sense of an appropriate analog of the reflection property 1. In practice, most meaningful assertions must already be treated probabilistically, and very little is lost by allowing some sentences to have probabilities intermediate between 0 and 1.
Another paper showing an application of this result to set theory is forthcoming.
March Newsletter
|
||||
|
Upcoming MIRI Research Workshops
From November 11-18, 2012, we held (what we now call) the 1st MIRI Workshop on Logic, Probability, and Reflection. This workshop had four participants:
- Eliezer Yudkowsky (MIRI)
- Paul Christiano (UC Berkeley)
- Marcello Herreshoff (Google)
- Mihály Bárász (Google)
The participants worked on the foundations of probabilistic reflective reasoning. In particular, they showed that a careful formalization of probabilistic logic can circumvent many classical paradoxes of self-reference. Applied to metamathematics, this framework provides (what seems to be) the first definition of truth which is expressive enough for use in reflective reasoning. Applied to set theory, this framework provides an implementation of probabilistic set theory based on unrestricted comprehension which is nevertheless powerful enough to formalize ordinary mathematical reasoning (in contrast with similar fuzzy set theories, which were originally proposed for this purpose but later discovered to be incompatible with mathematical induction).
These results suggest a similar approach may be used to work around Löb’s theorem, but this has not yet been explored. This work will be written up over the coming months.
In the meantime, MIRI is preparing for the 2nd MIRI Workshop on Logic, Probability, and Reflection, to take place from April 3-24, 2013. This workshop will be broken into two sections. The first section (Apr 3-11) will bring together the 1st workshop’s participants and 8 additional participants:
- Stuart Armstrong (Oxford University)
- Daniel Dewey (Oxford University)
- Benja Fallenstein (University of Vienna)
- Patrick LaVictoire (UW Madison)
- Jacob Steinhardt (Stanford University)
- Qiaochu Yuan (UC Berkeley)
- Andrew Critch (UC Berkeley)
- Jacob Taylor (Stanford)
The second section (Apr 12-24) will consist solely of the 4 participants from the 1st workshop.
Participants of this 2nd workshop will continue to work on the foundations of reflective reasoning, for example Gödelian obstacles to reflection, and decision algorithms for reflective agents (e.g. TDT).
Additional MIRI research workshops are also tentatively planned for the summer and fall of 2013.
Update: An early draft of the paper describing the first result from the 1st workshop is now available here.
Welcome to Intelligence.org
Welcome to the new home for the Machine Intelligence Research Institute (MIRI), formerly called “The Singularity Institute.”
The new design (from Katie Hartman, who also designed the new site for CFAR) reflects our recent shift in focus from “movement-building” to technical research. Our research and our research advisors are featured prominently on the home page, and our network of research associates are included on the Team page.
Getting involved is also clearer, with easy-to-find pages for applying to be a volunteer, an intern, a visiting fellow, or a research fellow.
Our About page hosts things like our transparency page, our top contributors list, our About page hosts things like our transparency page, our top contributors list, our new press kit, and our archive of all Singularity Summit talk videos, audio, and transcripts from 2006-2012. (The Summit was recently acquired by Singularity University.)
Follow our blog to keep up with the latest news and analyses. Recent analyses include Yudkowsky on Logical Uncertainty and Yudkowsky on “What Can We Do Now?”
We’ll be adding additional content in the next few months, so stay tuned!
