Re: We Can't Fool the Super Intelligence

From: Sebastian Hagen (sebastian_hagen@gmx.de)
Date: Mon Jun 28 2004 - 07:17:08 MDT


Thomas Buckner wrote:
> *I still stick to my rule 'Never limit your souces of information' because it has never, never let
> me down.
> [...]
> *And I would still be first in line for that chip because I think it would be safe ***as long as
> it had no wireless connectivity or other way for malware, viruses or hackers to mess with my
> brain!*** I think that would be the real source of problems. On the other hand, perhaps we will
> soon be vulnerabe to that even without consciously adding a chip. Uh oh... Can open, worms
> everywhere...
Afaik humans are intrinsically vulnerable. Certain parasitic memes already seem
to work pretty well; if you count religions, some have been doing well for millenia.
Sufficiently intelligent minds would in all likelihood be masters at memetic
warfare; the current parasitic memes are probably rather harmless in comparison
to what a Power could conceive of.
The ai-box experiments <http://yudkowsky.net/essays/aibox.html> may be of
relevance here.
To "Never limit your sources of information" seems like a good idea as long as
you don't have any (direct or indirect) contact to minds much more intelligent
than your own.
Exposing your own mind to memes created by a Power that you don't trust seems
like a very bad idea to me.

Sebastian Hagen



This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT