From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Fri Jul 22 2005 - 10:35:38 MDT
Michael Vassar wrote:
> For what it's worth, our single data point seems to indicate that there
> is at least a reasonably good chance that a slightly augmented human in
> Venezuela, or at least in Brian and Sabine Atkins house (?) would turn
> Friendly after initially pursuing UnFriendly goals. I'm not sure how we
> should take that, but it seems to me that we are stuck depending on
> slightly augmented humans because UFAI is enough easier than FAI that
> there is no chance of an ordinary human being able to put together FAI
> before a world full of other ordinary humans gets to UFAI. For that
> reason, my position remains favorable to unknown augmented humans, at
> least if they can be communicated with. This conclusion is not my
> dispositional tendency, as I was very (and remain somewhat) concerned by
> that first example before he came around, but unless we want to freeze
> technological progression via MNT what choice have we got?
If you use non-radical techniques of human augmentation, you have serious
trouble beating literally every other human in the world. If you don't create
literally the smartest human ever, then you've just wasted who knows how many
years of research to accomplish something you could have done with decent
recruiting. Moreover, even if you can beat every other human in the world
after ten years, you have to beat them by a fairly sizable amount if your
nascent transhumans are to become true experts in less than the ten-year lead
time that has been usually found for the training of genius. No one makes a
"genius" contribution without ten years of work, is the usual figure I hear.
Let us say that my clock started ticking in 1996... how much faster are these
non-radically-augmented transhumans than me, that they can learn in only a
year or two how to use their intelligence for something other than defeating
itself, and accumulate all the knowledge they need? And how long did it take
you to produce them?
If you use radical techniques of human augmentation then who can guarantee
that even an initially benevolent human would remain so? The brain is very
easy to disturb. It has a balance, it grows up with a balance, and it has an
even harder time repairing that balance after puberty. Humans are not
designed for radical modification. Everything in the brain is adapted to work
in the context of everything else in the brain working the way it does now.
Even increasing the input from one area at the expense of others, can throw
off the balance enough to result in what we regard as significant neurological
damage.
-- Eliezer S. Yudkowsky http://intelligence.org/ Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT