From: Samantha Atkins (samantha@objectent.com)
Date: Mon Jul 08 2002 - 14:06:28 MDT
Mike & Donna Deering wrote:
> If we are talking about super intelligent problem solving software at
> the disposal of human intelligent humans we are getting into the science
> fiction area of technological capabilities coexisting with sociological
> situations that make no logical sense. Such as BLADE RUNNER or TOTAL
> RECALL.
>
> Mike.
How would a human or any group of humans possibly understand or
be able to guide something that thinks around 10^6 times faster
than they do? If the AI is not conscious/self-aware then does
it blindly take whatevery the humans ask it to do and
extrapolate from there? Or can we assume it is at least
reflective enough to have a fairly detailed model of humans and
what works for humans and a model of its own states (oops - gets
close to self awareness) and simulations of the outcome of
possible actions. If we don't assume some or all of these
things then it seems to me you would have the Three Wishes
problem. Figuring out what to ask for that would not have
unforseen tragic consequences would be a bear.
- samantha
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:40 MDT