From July 8-14, MIRI will host its 3rd Workshop on Logic, Probability, and Reflection. The focus of this workshop will be the Löbian obstacle to self-modifying systems.

Participants confirmed so far include:

• Andrew Critch (just finished his math PhD at UC Berkeley, now working at CFAR)
• Abram Demski (USC)
• Benja Fallenstein (Bristol U)
• Marcello Herreshoff (Google)
• Will Sawin (Princeton)
• Qioachu Yuan (UC Berkeley)
• Eliezer Yudkowsky (MIRI)

If you have a strong mathematics background and might like to attend this workshop, it’s not too late to apply! And even if this workshop doesn’t fit your schedule, please do apply, so that we can notify you of other workshops (long before they are announced publicly).

Information on past workshops:

• Our 1st Workshop (Nov. 11-18, 2012; 4 participants) resulted in Christiano’s probabilistic logic, an attack on the Löbian obstacle for self-modifying systems.
  
• Our 2nd Workshop (Apr. 3-24, 2013; 12 participants coming in and out) resulted in (1) some as-yet unpublished progress on Christiano’s probabilistic logic, (2) some progress on program equilibrium recorded in LaVictoire et al. (2013), and some progress on the Löbian obstacle resulting in Yudkowsky & Herreschoff (2013).

The post MIRI’s July 2013 Workshop appeared first on Machine Intelligence Research Institute.

Our new Research page has launched!

Our previous research page was a simple list of articles, but the new page describes the purpose of our research, explains four categories of research to which we contribute, and highlights the papers we think are most important to read.

We’ve also released drafts of two new research articles.

Tiling Agents for Self-Modifying AI, and the Löbian Obstacle (discuss it here), by Yudkowsky and Herreshoff, explains one of the key open problems in MIRI’s research agenda:

We model self-modification in AI by introducing “tiling” agents whose decision systems will approve the construction of highly similar agents, creating a repeating pattern (including similarity of the offspring’s goals). Constructing a formalism in the most straightforward way produces a Gödelian difficulty, the “Löbian obstacle.” By technical methods we demonstrates the possibility of avoiding this obstacle, but the underlying puzzles of rational coherence are thus only partially addressed. We extend the formalism to partially unknown deterministic environments, and show a very crude extension to probabilistic environments and expected utility; but the problem of finding a fundamental decision criterion for self-modifying probabilistic agents remains open.

Robust Cooperation in the Prisoner’s Dilemma: Program Equilibrium via Provability Logic (discuss it here), by LaVictoire et al., explains some progress in program equilibrium made by MIRI research associate Patrick LaVictoire and several others during MIRI’s April 2013 workshop:

Rational agents defect on the one-shot prisoner’s dilemma even though mutual cooperation would yield higher utility for both agents. Moshe Tennenholtz showed that if each program is allowed to pass its playing strategy to all other players, some programs can then cooperate on the one-shot prisoner’s dilemma. Program equilibria is Tennenholtz’s term for Nash equilibria in a context where programs can pass their playing strategies to the other players.

One weakness of this approach so far has been that any two programs which make different choices cannot “recognize” each other for mutual cooperation, even if they are functionally identical. In this paper, provability logic is used to enable a more flexible and secure form of mutual cooperation.

Participants of MIRI’s April workshop also made progress on Christiano’s probabilistic logic (an attack on the Löbian obstacle), but that work is not yet ready to be released.

We’ve also revamped the Get Involved page, which now includes an application form for forthcoming workshops. If you might like to work with MIRI on some of its open research problems sometime in the next 18 months, please apply! Likewise, if you know someone who might enjoy attending such a workshop, please encourage them to apply.

The post New Research Page and Two New Articles appeared first on Machine Intelligence Research Institute.

MIRI was founded in 2000 on the premise that creating¹ Friendly AI might be a particularly efficient way to do as much good as possible.

Some developments since then include:

• The field of “effective altruism” — trying not just to do good but to do as much good as possible² — has seen more publicity and better research than ever before, in particular through the work of GiveWell, the Center for Effective Altruism, the philosopher Peter Singer, and the community at Less Wrong.³
• In his recent PhD dissertation, Nick Beckstead has clarified the assumptions behind the claim that shaping the far future (e.g. via Friendly AI) is overwhelmingly important.
• Due to research performed by MIRI, the Future of Humanity Institute (FHI), and others, our strategic situation with regard to machine superintelligence is more clearly understood, and FHI’s Nick Bostrom has organized much of this work in a forthcoming book.⁴
• MIRI’s Eliezer Yudkowsky has begun to describe in more detail which open research problems constitute “Friendly AI research,” in his view.

Given these developments, we are in a better position than ever before to assess the value of Friendly AI research as effective altruism.

Still, this is a difficult question. It is challenging enough to evaluate the cost-effectiveness of anti-malaria nets or direct cash transfers. Evaluating the cost-effectiveness of attempts to shape the far future (e.g. via Friendly AI) is even more difficult than that. Hence, this short post sketches an argument that can be given in favor of Friendly AI research as effective altruism, to enable future discussion, and is not intended as a thorough analysis.

An argument for Friendly AI research as effective altruism

Beckstead (2013) argues⁵ for the following thesis:

From a global perspective, what matters most (in expectation) is that we do what is best (in expectation) for the general trajectory along which our descendants develop over the coming millions, billions, and trillions of years.

Why think this? Astronomical facts suggest that humanity (including “post-humanity”) could survive for billions or trillions of years (Adams 2008), and could thus produce enormous amounts of good.⁶ But the value produced by our future depends on our development trajectory. If humanity destroys itself with powerful technologies in the 21st century, then nearly all that future value is lost. And if we survive but develop along a trajectory dominated by conflict and poor decisions, then the future could be much less good than if our trajectory is dominated by altruism and wisdom. Moreover, some of our actions today can have “ripple effects”⁷ which determine the trajectory of human development, because many outcomes are path-dependent. Hence, actions which directly or indirectly precipitate particular trajectory changes (e.g. mitigating existential risks) can have vastly more value (in expectation) than actions with merely proximate benefits (e.g. saving the lives of 20 wild animals). Beckstead calls this the “rough future-shaping argument.”

If we accept the normative assumptions lurking behind this argument (e.g. risk neutrality; see Beckstead’s dissertation), then the far future is enormously valuable (if it goes at least as well on average as the past century), and existential risk reduction is much more important than producing proximate benefits (e.g. global health, poverty reduction) or speeding up development (which could in fact increase existential risks, and even if it doesn’t, has lower expected value than existential risk reduction).

However, Beckstead’s conclusion is not necessarily that existential risk reduction should be our global priority, because

there may be other ways to have a large, persistent effect on the far future without reducing existential risk… Some persistent changes in values and social norms could make the future [some fraction] better or worse… Sure, succeeding in preventing an existential catastrophe would be better than making a smaller trajectory change, but creating a small positive trajectory change may be significantly easier.

Instead, Beckstead’s arguments suggest that “what matters most for shaping the far future is producing positive trajectory changes and avoiding negative ones.” Existential risk reduction is one important kind of positive trajectory change that could turn out to be the intervention with the highest expected value.

One important clarification is in order. It could turn out to be that working toward proximate benefits or development acceleration does more good than “direct” efforts for trajectory change, if working toward proximate benefits or development acceleration turns out to have major ripple effects which produce important trajectory change. For example, perhaps an “ordinary altruistic effort” like solving India’s iodine deficiency problem would cause there to be thousands of “extra” world-class elite thinkers two generations from now, which could increase humanity’s chances of intelligently navigating the crucial 21st century and spreading to the stars. (I don’t think this is likely; I suggest it merely for illustration.)

For the sake of argument, suppose you agree with Beckstead’s core thesis that “what matters most (in expectation) is that we do what is best (in expectation) for the general trajectory along which our descendants develop.” Suppose you also think, as I do, that machine superintelligence is probably inevitable.⁸

In that case, you might think that Friendly AI research is a uniquely foreseeable and impactful way to shape the far future in an enormously positive way, because “our effects on the far future must almost entirely pass through our effects on the development of machine superintelligence.” All other developing trends might be overridden by the overwhelming effectiveness of machine superintelligence — and specifically, by the values that were (explicitly or implicitly, directly or indirectly) written into the machine superintelligence(s).

If that’s right, our situation is a bit like sending an interstellar probe to colonize distant solar systems before they recede beyond the cosmological horizon and can thus never be reached from Earth again due to the expansion of the universe. Anything on Earth that doesn’t affect the content of the probe will have no impact on those solar systems. (See also this comment.)

Potential defeaters

The rough argument above — in favor of Friendly AI research as an efficient form of effective altruism — deserves to be “fleshed out” in more detail.⁹

Potential defeaters should also be examined:

• Perhaps we ought to reject one or more of the normative assumptions behind Beckstead’s rough future-shaping argument.
• Perhaps it’s not true that “our effects on the far future must almost entirely pass through our effects on the development of machine superintelligence.”
• Perhaps Friendly AI research is not (today) a particularly efficient way to positively affect the development of machine superintelligence. Competing interventions may include: (1) AI risk strategy research, (2) improving technological forecasting, (3) improving science in general, (4) improving and spreading effective altruism and rationality, and (5) many others.

In future blog posts, members of the effective altruist community (including myself) will expand on the original argument and examine potential defeaters.

Notes

My thanks to those who provided feedback on this post: Carl Shulman, Nick Beckstead, Jonah Sinick, and Eliezer Yudkowsky.

¹ In this post, I talk about the value of humanity in general creating Friendly AI, though MIRI co-founder Eliezer Yudkowsky usually talks about MIRI in particular — or at least, a functional equivalent — creating Friendly AI. This is because I am not as confident as Yudkowsky that it is best for MIRI to attempt to build Friendly AI. When updating MIRI’s bylaws in early 2013, Yudkowsky and I came to a compromise on the language of MIRI’s mission statement, which now reads: “[MIRI] exists to ensure that the creation of smarter-than-human intelligence has a positive impact. Thus, the charitable purpose of [MIRI] is to: (a) perform research relevant to ensuring that smarter-than-human intelligence has a positive impact; (b) raise awareness of this important issue; (c) advise researchers, leaders and laypeople around the world; and (d) as necessary, implement a smarter-than-human intelligence with humane, stable goals” (emphasis added). My own hope is that it will not be necessary for MIRI (or a functional equivalent) to attempt to build Friendly AI itself. But of course I must remain open to the possibility that this will be the wisest course of action as the first creation of AI draws nearer. There is also the question of capability: few people think that a non-profit research organization has much chance of being the first to build AI. I worry, however, that the world’s elites will not find it fashionable to take this problem seriously until the creation of AI is only a few decades away, at which time it will be especially difficult to develop the mathematics of Friendly AI in time, and humanity will be forced to take a gamble on its very survival with powerful AIs we have little reason to trust.

² One might think of effective altruism as a straightforward application of decision theory to the subject of philanthropy. Philanthropic agents of all kinds (individuals, groups, foundations, etc.) ask themselves: “How can we choose philanthropic acts (e.g. donations) which (in expectation) will do as much good as possible, given what we care about?” The consensus recommendation for all kinds of choices under uncertainty, including philanthropic choices, is to maximize expected utility (Chater & Oaksford 2012; Peterson 2004; Stein 1996; Schmidt 1998:19). Different philanthropic agents value different things, but decision theory suggests that each of them can get the most of what they want if they each maximize their expected utility. Choices which maximize expected utility are in this sense “optimal,” and thus another term for effective altruism is “optimal philanthropy.” Note that effective altruism in this sense is not too dissimilar from earlier approaches to philanthropy, including high-impact philanthropy (making “the biggest difference possible, given the amount of capital invested“), strategic philanthropy, effective philanthropy, and wise philanthropy. Note also that effective altruism does not say that a philanthropic agent should specify complete utility and probability functions over outcomes and then compute the philanthropic act with the highest expected utility — that is impractical for bounded agents. We must keep in mind the distinction between normative, descriptive, and prescriptive models of decision-making (Baron 2007): “normative models tell us how to evaluate… decisions in terms of their departure from an ideal standard. Descriptive models specify what people in a particular culture actually do and how they deviate from the normative models. Prescriptive models are designs or inventions, whose purpose is to bring the results of actual thinking into closer conformity to the normative model.” The prescriptive question — about what bounded philanthropic agents should do to maximize expected utility with their philanthropic choices — tends to be extremely complicated, and is the subject of most of the research performed by the effective altruism community.

³ See, for example: Efficient Charity, Efficient Charity: Do Unto Others, Politics as Charity, Heuristics and Biases in Charity, Public Choice and the Altruist’s Burden, On Charities and Linear Utility, Optimal Philanthropy for Human Beings, Purchase Fuzzies and Utilons Separately, Money: The Unit of Caring, Optimizing Fuzzies and Utilons: The Altruism Chip Jar, Efficient Philanthropy: Local vs. Global Approaches, The Effectiveness of Developing World Aid, Against Cryonics & For Cost-Effective Charity, Bayesian Adjustment Does Not Defeat Existential Risk Charity, How to Save the World, and What is Optimal Philanthropy?

⁴ I believe Beckstead and Bostrom have done the research community an enormous service in creating a framework, a shared language, for discussing trajectory changes, existential risks, and machine superintelligence. When discussing these topics with my colleagues, it has often been the case that the first hour of conversation is spent merely trying to understand what the other person is saying — how they are using the terms and concepts they employ. Beckstead’s and Bostrom’s recent work should enable clearer and more efficient communication between researchers, and therefore greater research productivity. Though I am not aware of any controlled, experimental studies on the effect of shared language on research productivity, a shared language is widely considered to be of great benefit for any field of research, and I shall provide a few examples of this claim which appear in print. Fuzzi et al. (2006): “The use of inconsistent terms can easily lead to misunderstandings and confusion in the communication between specialists from different [disciplines] of atmospheric and climate research, and may thus potentially inhibit scientific progress.” Hinkel (2008): “Technical languages enable their users, e.g. members of a scientific discipline, to communicate efficiently about a domain of interest.” Madin et al. (2007): “terminological ambiguity slows scientific progress, leads to redundant research efforts, and ultimately impedes advances towards a unified foundation for ecological science.”

⁵ In addition to Beckstead’s thesis, see also A Proposed Adjustment to the Astronomical Waste Argument.

⁶ Beckstead doesn’t mention this, but I would like to point out that moral realism is not required for Beckstead’s arguments to go through. In fact, I generally accept Beckstead’s arguments even though most philosophers would not consider me a moral realist, though to some degree that is a semantic debate (Muehlhauser 2011; Joyce 2012). If you’re a moral realist and you believe your intuitive moral judgments are data about what is morally true, then Beckstead’s arguments (if successful) have something to say about what is morally true, and about what you should do if you want to act in morally good ways. If you’re a moral anti-realist but you think your intuitive judgments are data about what you value — or about what you would value if you had more time to think about your values and how to resolve the contradictions among them — then Beckstead’s arguments (if successful) have something to say about what you value, and about what you should do if you want to help achieve what you value.

⁷ Karnofsky calls these “flow-through effects.”

⁸ See Bostrom (forthcoming) for an extended argument. Perhaps the most likely defeater for machine superintelligence is that global catastrophe may halt scientific progress before human-level AI is created.

⁹ Beckstead, in personal communication, suggested (but didn’t necessarily endorse) the following formalization of the rough argument sketched in the main text of the blog post: “(1) To a first approximation, the future of humanity is all that matters. (2) To a much greater extent than anything else, the future of humanity is highly sensitive to how machine intelligence unfolds. (3) Therefore, there is a very strong presumption in favor of working on any project which makes machine intelligence unfold in a better way. (4) FAI research is the most promising route to making machine intelligence unfold in a better way. (5) Therefore, there is a very strong presumption in favor of doing FAI research.” Beckstead (2013) examines the case for (1). Bostrom (forthcoming), in large part, examines the case for (2). Premise (3) informally follows from (1) and (2), and the conclusion (5) informally follows from (3) and (4). Premise (4) appears to me to be the most dubious part of the argument, and the least explored in the extant literature.

The post Friendly AI Research as Effective Altruism appeared first on Machine Intelligence Research Institute.

Greetings From the Executive Director Dear friends, It’s been a busy month! Mostly, we’ve been busy publishing things. As you’ll see below, Singularity Hypotheses has now been published, and it includes four chapters by MIRI researchers or research associates. We’ve also published two new technical reports — one on decision theory and another on intelligence explosion microeconomics — and several new blog posts analyzing various issues relating to the future of AI. Finally, we added four older articles to the research page, including Ideal Advisor Theories and Personal CEV (2012). In our April newsletter we spoke about our April 11th party in San Francisco, celebrating our relaunch as the Machine Intelligence Research Institute and our transition to mathematical research. Additional photos from that event are now available as a Facebook photo album. We’ve also uploaded a video from the event, in which I spend 2 minutes explaining MIRI’s relaunch and some tentative results from the April workshop. After that, visiting researcher Qiaochu Yuan spends 4 minutes explaining one of MIRI’s core research questions: the Löbian obstacle to self-modifying systems. Some of the research from our April workshop will be published in June, so if you’d like to read about those results right away, you might like to subscribe to our blog. Cheers! Luke Muehlhauser Executive Director Intelligence Explosion Microeconomics Our largest new publication this month is Yudkowsky’s 91-page Intelligence Explosion Microeconomics (discuss here). In this article, Yudkowsky takes some initial steps toward tackling the key quantitative issue in the intelligence explosion, “reinvestable returns on cognitive investments”: what kind of returns can you get from an investment in cognition, can you reinvest it to make yourself even smarter, and does this process die out or blow up? The article can be thought of as a compact and hopefully more coherent successor to the AI Foom Debate of 2009, featuring Yudkowsky and GMU economist Robin Hanson. Here is the abstract: I. J. Good’s thesis of the ‘intelligence explosion’ is that a sufficiently advanced machine intelligence could build a smarter version of itself, which could in turn build an even smarter version of itself, and that this process could continue enough to vastly exceed human intelligence. As Sandberg (2010) correctly notes, there are several attempts to lay down return-on-investment formulas intended to represent sharp speedups in economic or technological growth, but very little attempt has been made to deal formally with I. J. Good’s intelligence explosion thesis as such. I identify the key issue as returns on cognitive reinvestment — the ability to invest more computing power, faster computers, or improved cognitive algorithms to yield cognitive labor which produces larger brains, faster brains, or better mind designs. There are many phenomena in the world which have been argued as evidentially relevant to this question, from the observed course of hominid evolution, to Moore’s Law, to the competence over time of machine chess-playing systems, and many more. I go into some depth on the sort of debates which then arise on how to interpret such evidence. I propose that the next step forward in analyzing positions on the intelligence explosion would be to formalize return-on-investment curves, so that each stance can say formally which possible microfoundations they hold to be falsified by historical observations already made. More generally, I pose multiple open questions of ‘returns on cognitive reinvestment’ or ‘intelligence explosion microeconomics’. Although such questions have received little attention thus far, they seem highly relevant to policy choices affecting the outcomes for Earth-originating intelligent life. The dedicated mailing list will be small and restricted to technical discussants: apply for it here. When Will AI Be Created? In part, intelligence explosion microeconomics seeks to answer the question “How quickly will human-level AI self-improve to become superintelligent?” Another major question in AI forecasting is, of course: “When will we create human-level AI?” This is another difficult question, and Luke Muehlhauser surveyed those difficulties in a recent (and quite detailed) blog post: When Will AI Be Created? He concludes: Given these considerations, I think the most appropriate stance on the question “When will AI be created?” is something like this: “We can’t be confident AI will come in the next 30 years, and we can’t be confident it’ll take more than 100 years, and anyone who is confident of either claim is pretending to know too much.” How confident is “confident”? Let’s say 70%. That is, I think it is unreasonable to be 70% confident that AI is fewer than 30 years away, and I also think it’s unreasonable to be 70% confident that AI is more than 100 years away. This statement admits my inability to predict AI, but it also constrains my probability distribution over “years of AI creation” quite a lot. I think the considerations above justify these constraints on my probability distribution, but I haven’t spelled out my reasoning in great detail. That would require more analysis than I can present here. But I hope I’ve at least summarized the basic considerations on this topic, and those with different probability distributions than mine can now build on my work here to try to justify them. Muehlhauser also explains four methods for reducing our uncertainty about AI timelines: explicit quantification, leveraging aggregation, signposting the future, and decomposing the phenomena. As it turns out, you can participate in the first two methods for improving our AI forecasts by signing up for GMU’s DAGGRE program. Muehlhauser himself has signed up. Muehlhauser also wrote a 400-word piece on the difficulty of AI forecasting for Quartz magazine: Robots will take our jobs, but it’s hard to say when. Here’s a choice quote: We’ve had the computing power of a honeybee’s brain for quite a while now, but that doesn’t mean we know how to build tiny robots that fend for themselves outside the lab, find their own sources of energy, and communicate with others to build their homes in the wild.   Singularity Hypothesis Published Singularity Hypotheses: A Scientific and Philosophical Assessment has now been published by Springer, in hardcover and ebook forms. The book contains 20 chapters about the prospect of machine superintelligence, including 4 chapters by MIRI researchers and research associates: Intelligence Explosion: Evidence and Import by Luke Muehlhauser and Anna Salamon Intelligence Explosion and Machine Ethics by Luke Muehlhauser and Louie Helm Friendly Artificial Intelligence by Eliezer Yudkowsky, a shortened version of Yudkowsky (2008) Artificial General Intelligence and the Human Mental Model by Roman Yampolskiy and (MIRI research associate) Joshua Fox For more details, see the blog post. Timeless Decision Theory Paper Published During his time as a research fellow for MIRI, Alex Altair wrote an article on Timeless Decision Theory (TDT) that has now been published: “A Comparison of Decision Algorithms on Newcomblike Problems.” Altair’s article is both more succinct and also more precise in its formulation of TDT than Yudkowsky’s earlier paper “Timeless Decision Theory.” Thus, Altair’s paper should serve as a handy introduction to TDT for philosophers, computer scientists, and mathematicians, while Yudkowsky’s paper remains required reading for anyone interested to develop TDT further, for it covers more ground than Altair’s paper. For a gentle introduction to the entire field of normative decision theory (including TDT), see Muehlhauser and Williamson’s Decision Theory FAQ. AGI Impacts Experts and Friendly AI Experts In AGI Impacts Experts and Friendly AI Experts, Luke Muehlhauser explains the two types of experts MIRI hopes to cultivate. AGI impacts experts develop skills related to predicting technological development (e.g. building computational models of AI development or reasoning about intelligence explosion microeconomics), predicting AGI’s likely impacts on society, and identifying which interventions are most likely to increase humanity’s chances of safely navigating the creation of AGI. For overviews, see Bostrom & Yudkowsky (2013); Muehlhauser & Salamon (2013). Friendly AI experts develop skills useful for the development of mathematical architectures that can enable AGIs to be trustworthy (or “human-friendly”). This work is carried out at MIRI research workshops and in various publications, e.g. Christiano et al. (2013); Hibbard (2013). Note that the term “Friendly AI” was selected (in part) to avoid the suggestion that we understand the subject very well — a phrase like “Ethical AI” might sound like the kind of thing one can learn a lot about by looking it up in an encyclopedia, but our present understanding of trustworthy AI is too impoverished for that. For more details on which skills these kinds of experts should develop, read the blog post. MIRI’s Mission in Five Theses and Two Lemmas Yudkowsky sums up MIRI’s research mission in the blog post Five theses, two lemmas, and a couple of strategic implications. The five theses are: Intelligence explosion Orthogonality of intelligence and goals Convergent instrumental goals Complexity of value Fragility of value According to Yudkowsky, these theses imply two important lemmas: Indirect normativity Large bounded extra difficulty of Friendliness In turn, these two lemmas have two important strategic implications: We have a lot of work to do on things like indirect normativity and stable self-improvement. At this stage a lot of this work looks really foundational — that is, we can’t describe how to do these things using infinite computing power, let alone finite computing power.  We should get started on this work as early as possible, since basic research often takes a lot of time. There needs to be a Friendly AI project that has some sort of boost over competing projects which don’t live up to a (very) high standard of Friendly AI work — a project which can successfully build a stable-goal-system self-improving AI, before a less-well-funded project hacks together a much sloppier self-improving AI. Giant supercomputers may be less important to this than being able to bring together the smartest researchers… but the required advantage cannot be left up to chance. Leaving things to default means that projects less careful about self-modification would have an advantage greater than casual altruism is likely to overcome. For more details on the theses and lemmas, read the blog post and its linked articles. Our Final Invention available for preorder James Barrat, a documentary filmmaker for National Geographic, Discovery, PBS, and other broadcasters, has written a wonderful new book about the intelligence explosion called Our Final Invention: Artificial Intelligence and the End of the Human Era. It will be released October 1st, and is available for pre-order on Amazon. Here are some blurbs from people who have read an advance copy: “A hard-hitting book about the most important topic of this century and possibly beyond — the issue of whether our species can survive. I wish it was science fiction but I know it’s not.” —Jaan Tallinn, co-founder of Skype “The compelling story of humanity’s most critical challenge. A Silent Spring for the 21st Century.” —Michael Vassar, former MIRI president “Our Final Invention is a thrilling detective story, and also the best book yet written on the most important problem of the 21st century.” —Luke Muehlhauser, MIRI executive director “An important and disturbing book.” —Huw Price, co-founder, Cambridge University Center for the Study of Existential Risk MIRI Needs Advisors MIRI currently has a few dozen volunteer advisors on a wide range of subjects, but we need more! We’re especially hoping for additional advisors in mathematical logic, theoretical computer science, artificial intelligence, economics, and game theory. If you sign up, we will occasionally ask you questions, or send you early drafts of upcoming writings for feedback. We don’t always want technical advice (“Well, you can do that with a relativized arithmetical hierarchy…”); often, we just want to understand how different groups of experts respond to our writing (“The tone of this paragraph rubs me the wrong way because…”). Even if you don’t have much time to help, please sign up! We will of course respect your own limits on availability. Featured Volunteer – Florian Blumm Florian Bumm has been one of our most active translators. Florian translates materials from English into his native tongue, German. Historically a software engineer, he is now on a traveling vacation in Bolivia progressively extended by remote contract labor, which he has found conducive to his volunteering for MIRI. After leaving a position as a Java engineer for a financial services company, he has decided that he would rather contribute directly to a cause of some sort, and has determined that there is nothing more important than mitigating existential risks from artificial intelligence. Thanks, Florian! To join Florian and dozens of other volunteers, visit MIRIvolunteers.org.

You’re receiving this because you have subscribed to content from the Machine Intelligence Research Institute. Copyright © Machine Intelligence Research Institute, All rights reserved.

The post MIRI May Newsletter: Intelligence Explosion Microeconomics and Other Publications appeared first on Machine Intelligence Research Institute.

In When Will AI Be Created?, I referred to a bloggingheads.tv conversation between Eliezer Yudkowsky and Scott Aaronson. A transcript of that dialogue is now available, thanks to MIRI volunteers Ethan Dickinson, Daniel Kokotajlo, and Rick Schwall.

See also the transcript for a bloggingheads.tv conversation between Eliezer Yudkowsky and Massimo Pigliucci.

To join these volunteers in assisting our cause, visit MIRIvolunteers.org!

The post New Transcript: Yudkowsky and Aaronson appeared first on Machine Intelligence Research Institute.

GMU’s DAGGRE program, one of five teams participating in ACE, recently announced a transition from geopolitical forecasting to science & technology forecasting:

DAGGRE will continue, but it will transition from geo-political forecasting to science and technology (S&T) forecasting to better use its combinatorial capabilities. We will have a brand new shiny, friendly and informative interface co-designed by Inkling Markets, opportunities for you to provide your own forecasting questions and more!

Another exciting development is that our S&T forecasting prediction market will be open to everyone in the world who is at least eighteen years of age. We’re going global!

If you want to help improve humanity’s ability to forecast important technological developments like AI, please register for DAGGRE’s new S&T prediction website here.

I did.

The post Sign up for DAGGRE to improve science & technology forecasting appeared first on Machine Intelligence Research Institute.