From: Michael Roy Ames (michaelroyames@hotmail.com)
Date: Thu Oct 17 2002 - 20:35:28 MDT
Reason,
In order to answer to your question:
> What's the mechanism by which the big brain obtains
> information on what the wishes of the little brain
> are without contaminating the little brain by
> requesting information or passively making information
> available to let the little brain know it should be
> formulating wishes?
... I need to explain a viewpoint... My ~take~ on Eliezer's phrasing: 'what you
are doing ceases to be "argument" and becomes "mind control"' has been to draw
an analogy of "transhuman convincing PD human" being like "PD human convincing
dog" (PD = Present Day). A PD human can model a dog's "mind" sufficiently well
to convince it to do almost anything willingly, given a little time. The same
would be true of a transhuman and a PD human, except more so. The subjects,
both dog and PD human, may know very well that they are being pursuaded of
something, but will happily go along with it, when presented with a sufficiently
good argument, or reason, or incentive. We currently cannot read a dog's mind,
but we nevertheless can control its mind, to a non-trivial extent. Therefore,
getting around to answering your question, a transhuman may not need to know
one-whole-heck of a lot about a particular human, or even question them in a
direct fashion, to figure out what thier wishes are (to a non-trivial extent).
On the other hand a transhuman could just 'read' the information directly from
the PD human's brain using advanced non-invasive analytical tools, without any
conversational interaction at all.
And IRT your:
> I don't see a level at which any minimal conversation
> or interaction to determine wishes is not mind
> control of those wishes (in some way meaningful to
> the big brain) if you define mind control
> (advertantly or otherwise) as you did in your
> previous e-mail.
I believe you are (or will be) entirely correct in this opinion. Any
interaction of an advanced transhuman with a PD human could easily involve
extensive, detailed modelling of that human, and might intrinsicly include a
correspondingly high level of "mind control" or "manipulation" (or whatever will
be the latest jargon of that time) unless the transhuman makes meticulous
efforts to avoid it. Once one comes to believe that this will be the case, the
importance of Friendliness in a transhuman AI becomes overwhelmingly apparent.
Michael Roy Ames
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:41 MDT