Blog

Author: Scott Garrabrant

Finite Factored Sets

[mathjax] This is the edited transcript of a talk introducing finite factored sets. For most readers, it will probably be the best starting point for learning about factored sets. Video: (Lightly edited) slides: https://intelligence.org/files/Factored-Set-Slides.pdf     (Part 1, Title Slides)...

Saving Time

Note: This is a preamble to Finite Factored Sets, a sequence I’ll be posting over the next few weeks. This Sunday at noon Pacific time, I’ll be giving a Zoom talk (link) introducing Finite Factored Sets, a framework which I...

Subsystem Alignment

[mathjax]   You want to figure something out, but you don’t know how to do that yet. You have to somehow break up the task into sub-computations. There is no atomic act of “thinking”; intelligence must be built up of...

Embedded World-Models

  [mathjax] An agent which is larger than its environment can:   Hold an exact model of the environment in its head. Think through the consequences of every potential course of action. If it doesn’t know the environment perfectly, hold...

Embedded Agents

  [mathjax] Suppose you want to build a robot to achieve some real-world goal for you—a goal that requires the robot to learn for itself and figure out a lot of things that you don’t already know. ((This is part...

New paper: “Categorizing variants of Goodhart’s Law”

Goodhart’s Law states that “any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” However, this is not a single phenomenon. In Goodhart Taxonomy, I proposed that there are (at least) four different...