Re: ESSAY: How to deter a rogue AI by using your first-mover advantage

From: Stathis Papaioannou (
Date: Sat Aug 25 2007 - 22:33:48 MDT

On 25/08/07, Norman Noman <> wrote:

> If I wake up in the morning in what seems to be my room, then it probably is
> my room, and not a perfect copy of my room. If there was a TV show where
> people are drugged and kidnapped during the night and placed inside a
> perfect copy of their room, then the probability that my room was a copy
> would be slightly greater, and if you win a billion dollars on this game
> show if the first thing you do upon waking is yell "Q Q Q Q" then I might
> make that part of my morning routine, because even though the probability
> I'm on the show is tiny, the ratio of investment is so small, and the payoff
> is so big.
> The situation with the RAI is EXACTLY the same, and the fact that it
> involves AIs and computer simulations and universes is irrelevant.

Suppose a religious organisation announces that, when the technology
becomes available, they will create a simulation in which the faithful
are rewarded with heaven while atheists are punished in hell, unless
they genuinely convert or make a small donation to assist the
organisation in creating the simulation. What are you going to do?

Stathis Papaioannou

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:58 MDT