Re: ESSAY: How to deter a rogue AI by using your first-mover advantage

From: Stathis Papaioannou (
Date: Thu Aug 23 2007 - 08:54:51 MDT

On 23/08/07, Tom McCabe <> wrote:
> "It all might be a simulation" is a zero-knowledge
> hypothesis, since you can use it to explain any
> possible result. Do we refrain from eating cheesecake
> while balancing a glass of chocolate milk on our
> heads, because that might cause a shutdown of the
> simulation? No? Then why should the UFAI refrain from
> killing humans?

Yes: if it's a perfectly transparent (perfectly opaque?) simulation,
there is no way of testing any hypothesis you may come up with about
the simulators. Maybe the simulators are honest about the threat of
retribution, maybe they're just pretending, or maybe there are no
simulators. Each of these possibilities and every other possibility is
perfectly compatible with all available evidence, and there is no way
(by definition) of obtaining further evidence to help you decide which
is more likely. It would be like trying to guess God's shoe size in
the absence of any divine revelation.

Stathis Papaioannou

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:58 MDT