Blog

Category: Analysis

Robust Cooperation: A Case Study in Friendly AI Research

The paper “Robust Cooperation in the Prisoner’s Dilemma: Program Equilibrium via Provability Logic” is among the clearer examples of theoretical progress produced by explicitly FAI-related research goals. What can we learn from this case study in Friendly AI research? How...

How Big is the Field of Artificial Intelligence? (initial findings)

Co-authored with Jonah Sinick. How big is the field of AI, and how big was it in the past? This question is relevant to several issues in AGI safety strategy. To name just two examples: AI forecasting. Some people forecast...

From Philosophy to Math to Engineering

For centuries, philosophers wondered how we could learn what causes what. Some argued it was impossible, or possible only via experiment. Others kept hacking away at the problem, clarifying ideas like counterfactual and probability and correlation by making them more...

Russell and Norvig on Friendly AI

AI: A Modern Approach is by far the dominant textbook in the field. It is used in 1200 universities, and is currently the 22nd most-cited publication in computer science. Its authors, Stuart Russell and Peter Norvig, devote significant space to...

Richard Posner on AI Dangers

Richard Posner is a jurist, legal theorist, and economist. He is also the author of nearly 40 books, and is by far the most-cited legal scholar of the 20th century. In 2004, Posner published Catastrophe: Risk and Response, in which...

Mathematical Proofs Improve But Don’t Guarantee Security, Safety, and Friendliness

In 1979, Michael Rabin proved that his encryption system could be inverted — so as to decrypt the encrypted message — only if an attacker could factor n. And since this factoring task is computationally hard for any sufficiently large...