When asked their opinions about “human-level artificial intelligence” — aka “artificial general intelligence” (AGI)1 — many experts understandably reply that these terms haven’t yet been precisely defined, and it’s hard to talk about something that hasn’t been defined.2 In this post, I want to briefly outline an imprecise but useful “working definition” for intelligence we tend to use at MIRI. In a future post I will write about some useful working definitions for artificial general intelligence.
Imprecise definitions can be useful
Precise definitions are important, but I concur with Bertrand Russell that
[You cannot] start with anything precise. You have to achieve such precision… as you go along.
Physicist Milan Ćirković agrees, and gives an example:
The formalization of knowledge — which includes giving precise definitions — usually comes at the end of the original research in a given field, not at the very beginning. A particularly illuminating example is the concept of number, which was properly defined in the modern sense only after the development of axiomatic set theory in the… twentieth century.3
For a more AI-relevant example, consider the concept of a “self-driving car,” which has been given a variety of vague definitions since the 1930s. Would a car guided by a buried cable qualify? What about a modified 1955 Studebaker that could use sound waves to detect obstacles and automatically engage the brakes if necessary, but could only steer “on its own” if each turn was preprogrammed? Does that count as a “self-driving car”?
What about the “VaMoRs” of the 1980s that could avoid obstacles and steer around turns using computer vision, but weren’t advanced enough to be ready for public roads? How about the 1995 Navlab car that drove across the USA and was fully autonomous for 98.2% of the trip, or the robotic cars which finished the 132-mile off-road course of the 2005 DARPA Grand Challenge, supplied only with the GPS coordinates of the route? What about the winning cars of the 2007 DARPA Grand Challenge, which finished an urban race while obeying all traffic laws and avoiding collisions with other cars? Does Google’s driverless car qualify, given that it has logged more than 500,000 autonomous miles without a single accident under computer control, but still struggles with difficult merges and snow-covered roads?4
Our lack of a precise definition for “self-driving car” doesn’t seem to have hindered progress on self-driving cars very much.5 And I’m glad we didn’t wait to seriously discuss self-driving cars until we had a precise definition for the term.
Similarly, I don’t think we should wait for a precise definition of AGI before discussing the topic seriously. On the other hand, the term is useless if it carries no information. So let’s work our way toward a stipulative, operational definition for AGI. We’ll start by developing an operational definition for intelligence.
A definition for “intelligence”
Legg and Hutter (2007) found that definitions of intelligence converge toward the idea that “Intelligence measures an agent’s ability to achieve goals in a wide range of environments.” Let’s call this the “optimization power” concept of intelligence, because it measures an agent’s power to optimize the world according to its preferences.
I think this is a productive approach to the issue, since it identifies intelligence with externally measurable performance rather than with the details of how that performance might be achieved (e.g. via consciousness, brute force calculation, “complexity,” or something else). Moreover, it’s usually performance we care about: we tend to care most about whether an AI will perform well enough to replace human workers, or whether it will perform well enough improve its own abilities without human assistance, not whether it has some particular internal feature.6
Furthermore, the concept of optimization power allows us to compare the intelligence of different kinds of agents. As Albus (1991) said, “A useful definition of intelligence… should include both biological and machine embodiments, and these should span an intellectual range from that of an insect to that of an Einstein, from that of a thermostat to that of the most sophisticated computer system that could ever be built.”
I’d like to add one more consideration, though. What if two agents have roughly equal ability to optimize the world according to their preferences, but the second agent requires far more resources to do so? These agents have the same optimization power, but the first one seems to be optimizing more intelligently. So perhaps we could use “intelligence” to mean “optimization power divided by resources used” — what Yudkowsky called efficient cross-domain optimization.7
Other definitions8 have their merits, too. But at MIRI we find the concept of “efficient cross-domain optimization” sufficiently useful that it serves as our (still imprecise!) working definition for intelligence.
In a future post (Edit: here), I’ll discuss some useful working definitions for artificial general intelligence.
- I use the HLAI and AGI interchangeably, but lately I’ve been using AGI almost exclusively, because I’ve learned that many people in the AI community react negatively to any mention of “human-level” AI but have no objection to the concept of narrow vs. general intelligence. See also Ben Goertzel’s comments here. ↩
- Asked when he thought HLAI would be created, Pat Hayes (a past president of AAAI) replied: “I do not consider this question to be answerable, as I do not accept this (common) notion of ‘human-level intelligence’ as meaningful.” Asked the same question, AI scientist William Uther replied: “You ask a lot about ‘human level AGI’. I do not think this term is well defined,” while AI scientist Alan Bundy replied: “I don’t think the concept of ‘human-level machine intelligence’ is well formed.” ↩
- Sawyer (1943) gives another example: “Mathematicians first used the sign √-1, without in the least knowing what it could mean, because it shortened work and led to correct results. People naturally tried to find out why this happened and what √-1 really meant. After two hundreds years they succeeded.” Dennett (2013) makes a related comment: “Define your terms, sir! No, I won’t. That would be premature… My [approach] is an instance of nibbling on a tough problem instead of trying to eat (and digest) the whole thing from the outset… In Elbow Room, I compared my method to the sculptor’s method of roughing out the form in a block of marble, approaching the final surfaces cautiously, modestly, working by successive approximation.” ↩
- With self-driving cars, researchers did use many precise external performance measures (e.g. accident rates, speed, portion of the time they could run unassisted, frequency of getting stuck) to evaluate progress, as well as internal performance metrics (speed of search, bounded loss guarantees, etc.). Researchers could see that these bits of progress were in the right direction, even if their relative contribution long-term was unclear. And so it is with AI in general. AI researchers use many precise external and internal performance measures to evaluate progress, but it is difficult to know the relative contribution of these bits of progress toward the final goal of AGI. ↩
- Heck, we’ve had pornography for millennia and still haven’t been able to define it precisely. Encyclopedia entries for “pornography” often simply quote Justice Potter Stewart: “I shall not today attempt further to define the kinds of material I understand to be [pornography]… but I know it when I see it.” ↩
- We might care about whether machines are conscious in addition to being intelligent, but we already have a convenient term for that: consciousness. In particular, we might care about machine consciousness because the slow, plodding invention of AI may involve the creation and destruction of millions of partially-conscious near-AIs that are switched on, suffer for a while, and are then switched off — all while being unable to signal to us that they are suffering. This is especially likely if we remain unclear about the nature of consciousness for several more decades, and thus have no principled way (e.g. via nonperson predicates) to create intelligent machines that we know are not conscious (and are thus incapable of suffering). One of the first people to make this point clearly was Metzinger (2003), p. 621: “What would you say if someone came along and said, ‘Hey, we want to genetically engineer mentally retarded human infants! For reasons of scientific progress we need infants with certain cognitive and emotional deficits in order to study their postnatal psychological development—we urgently need some funding for this important and innovative kind of research!’ You would certainly think this was not only an absurd and appalling but also a dangerous idea. It would hopefully not pass any ethics committee in the democratic world. However, what today’s ethics committees don’t see is how the first machines satisfying a minimally sufficient set of constraints for conscious experience could be just like such mentally retarded infants. They would suffer from all kinds of functional and representational deficits too. But they would now also subjectively experience those deficits. In addition, they would have no political lobby—no representatives in any ethics committee.” Metzinger repeats the point in Metzinger (2010), starting on page 194. ↩
- Admittedly, this is still pretty vague. One step toward precision would be to propose a definition of intelligence as optimization power for some canonical distribution of possible preferences, over some canonical distribution of environments, with a penalty for resource use. The canonical preferences and canonical environments could be weighted toward preferences and environments relevant to our concerns: we care more about whether AIs can do science than whether they can paint abstract art, and we care more about whether they can achieve their goals in our solar system than whether they can achieve their goals inside a black hole. Also see Goertzel (2010)‘s “efficient pragmatic general intelligence.” ↩
- Hibbard (2011); Legg & Veness (2011); Wang (2008); Schaul et al. (2011); Dowe & Hernandez-Orallo (2012); Goertzel (2010); Adams et al. (2011). ↩
Did you like this post? You may enjoy our other Analysis posts, including:
