From: Samantha Atkins (samantha@objectent.com)
Date: Thu Apr 05 2001 - 03:24:53 MDT
"Eliezer S. Yudkowsky" wrote:
>
> James Higgins wrote:
> >
> > I'm not personally convinced that uploading a human first isn't the correct
> > answer. I'm rather inclined to trust a person more than an AI to map out
> > our destiny.
>
> The problem with uploading is that, chronologically, it arrives
> considerably after both artificial intelligence and military-grade
> nanotechnology. So unless you use a superintelligent AI as assistant,
> uploading can't plausibly come first. But if you can trust an AI to play
> assistant, then you can trust it to be Transition Guide (i.e., essentially
> the same order of risk in both cases).
>
Of course there are people interested in and working on the upload
scenario that do not believe you require either military-grade NT or an
sentient AI to
scan brains sufficiently to upload.
> I'm also more inclined to trust an AI more than a person... maybe even
> more than I'd trust myself, since I'm not designed for recursive
> self-improvement. Actually, I should amend that: After I've been around
> an AI and had a chance to chat with ver, then I expect to wind up
> justifiably trusting that AI's seed morality around as much as I'd trust a
> human seed morality, and I can also foresee the possibility of standing in
> the AI's presence and just being overawed by vis seed morality. Either
> way, I also expect to wind up trusting that AI's transcendence protocol to
> preserve morality significantly more than I'd trust a human-based
> transcendence protocol to preserve morality.
>
I don't see how this follows. If you upload a human who quickly
self-improves ver capabilities and becomes an SI and if
super-intelligence brings with it expanded moral/ethical understanding
then I see no reason this combination is less trustworthy than starting
from scratch and only putting in what you believe should be there in the
beginning. Yes a lot of evolved complicated behavior and conditioning
is not present in the AI. But some of that complicated behavior and
conditioning is also the bed of universal compassion and utter
Friendliness.
- samantha
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:36 MDT