From: Philip Sutton (Philip.Sutton@green-innovations.asn.au)
Date: Sun May 30 2004 - 06:10:35 MDT
Hi Eliezer,
You clearly think it's better to have just one FAI project to worry about.
But isn't this just engaging in fantasy thinking because there *isn't* just
one AGI project. What's the point in trying to theorise or act practically
in a situation that doesn't exist and cannot exisit unless somebody
successfully persuades all but one AGI projects to close down.
Knowing the huge personal and in some cases organisational
comitment behind the current multiple AGI projects I can't see their
number being reduced to one unless this is done by successful
forceable coercion. My understanding is that this latter action is not
something that you or the Singularity Institute are advocating or
intending to do.
Successful coercion could come from conventional human forces - but
most likely only in the context of *global* dictatorship or *gobal* anti-
AGI panic - or from the first super AGI destroying all other AGI projects.
It seems to me that there will be many AGI projects - at least for some
time. If there is any prospect that an UNfriendly AGI could get close to
take off then it seems to me that all other AGI projects will need to
cooperate to ensure that they are not eliminated and that the UN
friendly AGI is change or made ineffective.
If the first AGI project is Friendly, then it might be able to help
subsequent projects to be friendly too - most likely a better course of
action than setting out to destroy all other AGI projects.
So it seems to me that theorising should start with reality - that there
will be many AGI projects for some time and that a drastic reduction in
the number of projects and indeed the number of instantiations is only
likely to occur in very negative circumstances.
In this context I think it's worth theorising and then acting on the notion
of ensuring *slow* take-off - this avoids anyone having to go into panic
mode and allows a lot more learning from experience.
Cheers, Philip
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT