Re: Investing in FAI research: now vs. later

From: Matt Mahoney (
Date: Wed Feb 20 2008 - 19:07:56 MST

--- Nick Tarleton <> wrote:

> On Wed, Feb 20, 2008 at 5:54 PM, Matt Mahoney <> wrote:
> > The question is whether we become the machine that turns the earth into
> > paperclips, assuming it has some approximation of a copy of our memories,
> or
> > do we just die?
> Why would a paperclip maker, or a computronium maker, retain our memories?

The cost of doing so would be negligible, and it would benefit by having more
knowledge. Even if it didn't, it could still generate a plausible script and
you wouldn't know the difference.

> (Even if it did, I would find it little consolation.)

That is your present self talking. Knowing more, you might feel differently.

> > I don't believe the earth would be turned into paperclips because
> paperclip
> > makers would have less reproductive fitness than computronium makers.
> That
> > doesn't affect the basic issue, though.
> If the paperclip-maker takes off first and hard enough, fitness
> shouldn't matter. See

Thanks for the link. Yes, you're right. Evolution could be subverted by
cooperation. I would view that as a great disaster (although knowing more, I
might feel differently).

-- Matt Mahoney,

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:02 MDT