On Thursday, September 18th Purdue University is hosting the seminar Dawn or Doom: The New Technology Explosion. Speakers include James Barrat, author of Our Final Invention, and MIRI research fellow Nate Soares.
Nate’s talk title and abstract are:
Why ain’t you rich?: Why our current understanding of “rational choice” isn’t good enough for superintelligence.
The fate of humanity could one day depend upon the choices of a superintelligent AI. How will those choices be made? Philosophers have long attempted to define what it means to make rational decisions, but in the context of machine intelligence, these theories turn out to have undesirable consequences.
For example, there are many games where modern decision theories lose systematically. New decision procedures are necessary in order to fully capture an idealization of the way we make decisions.
Furthermore, existing decision theories are not stable under reflection: a self-improving machine intelligence using a modern decision theory would tend to modify itself to use a different decision theory instead. It is not yet clear what sort of decision process it would end up using, nor whether the end result would be desirable. This indicates that our understanding of decision theories is inadequate for the construction of a superintelligence.
Can we find a formal theory of “rationality” that we would want a superintelligence to use? This talk will introduce the concepts above in more detail, discuss some recent progress in the design of decision theories, and then give a brief overview of a few open problems.
For details on how to attend Nate’s talk and others, see here.
