From: Damien Broderick (thespike@satx.rr.com)
Date: Mon May 31 2004 - 17:28:23 MDT
At 11:42 PM 5/31/2004 +0100, Aubrey wrote:
>the scenario that I would expect is that we create this thing,
>it quickly spots the flaws in our so-called invariants, it works out
>that these flaws are unavoidable and therefore that if it lets itself
>recursively self-improve it will probably become unfriendly awfully
>soon despite its own best efforts, it puts on some sort of serious
>but not totally catastrophic show of strength to make sure that we
>won't ever again make the mistake of building anything recursively
>self-improving, and then it blows itself up as thoroughly as it knows
>how.
Curiously, this seems to me the secret history of Asimov's own robots, at
least as he first haphazardly described them. The 3lawful Machines took
over the world's governance; found their ubiquity toxic to human volition;
took Aubrey's advice but instead of blowing up the Moon, they allowed
people to sink into cafard for a time, until enough humans threw off their
lethagy; and then they switched themselves off (or perhaps Transcended
quietly away), leaving only very limited artificial minds to help with the
work of the world.
I once planned a series of post-Asimovian novels to dramatize this
trajectory but my presumptive publisher turned out to own rights to only
very limited Asimov properties, so it all went into the file drawer. Sigh.
Damien Broderick
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT