Re: Building a friendly AI from a "just do what I tell you" AI

From: Panu Horsmalahti (
Date: Thu Nov 22 2007 - 12:51:09 MST

Here's a FAI design:

First, there is an swarm AGIs of about the human intelligence level, with
the average human knowledge and task of creating a Friendly AI. The AGIs are
living in a virtual simulation without their knowledge, and the AGIs are
really slowly rewriting themselves to increase their intelligence. If humans
can code a FAI, then it must be possible for these AGIs too.

There is another computer that the AGIs don't know about, that sometimes
reads through the data from the AGI virtual world and creates a virtual
simulation (with the help of supercomputers) to test that FAI design. The
FAI has a really low probability to predict the possibility of a virtual
world simulation, so the probability of escaping is low. The computer does
some tasks trying to verify whether the FAI design works.

All this is run in another virtual layer with a computer observing possible
escapes from the virtual layer, and this layering can be made 100 times if

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:00 MDT