Posts By: Abram Demski

This is the conclusion of the Embedded Agency series. Previous posts:   Embedded Agents  —  Decision Theory  —  Embedded World-ModelsRobust Delegation  —  Subsystem Alignment     A final word on curiosity, and intellectual puzzles: I described an embedded agent, Emmy, and said that I don’t understand how she evaluates her options, models the world, models… Read more »

Because the world is big, the agent as it is may be inadequate to accomplish its goals, including in its ability to think. Because the agent is made of parts, it can improve itself and become more capable. Improvements can take many forms: The agent can make tools, the agent can make successor agents, or… Read more »

  Decision theory and artificial intelligence typically try to compute something resembling $$\underset{a \ \in \ Actions}{\mathrm{argmax}} \ \ f(a).$$ I.e., maximize some function of the action. This tends to assume that we can detangle things enough to see outcomes as a function of actions. For example, AIXI represents the agent and the environment as… Read more »