From: Ben Houston (email@example.com)
Date: Mon Jun 11 2001 - 20:18:15 MDT
The Law of Requisite Variety
As a possibly useful side-note I would point you to "the law of
requisite variety" of the cybernetician Ross Ashby. This law concerns
the relationship between to interacting systems. It states that for
system A to regulate (i.e. control) another system B, system A must have
at least the variety of system B. In other words the more adaptive
controls the less adaptive...
From: firstname.lastname@example.org [mailto:email@example.com] On Behalf
Of Durant Schoon
Sent: Monday, June 11, 2001 3:15 PM
Subject: Coercive Transhuman Memes and Exponential Cults
In March, Eliezer mention the notion of "coercive transhuman memes" when
discussing another issue:
> Date: Tue, 20 Mar 2001 05:02:56 -0500
> From: "Eliezer S. Yudkowsky" <firstname.lastname@example.org>
> To: email@example.com
> Subject: Re: How To Live In A Simulation
> What do I mean by a "commonsense" solution? I mean one in which
> second-order moral imperatives are applied with room for slack, rather
> than being absolute. By "second-order", I refer not to basic
> Friendliness, but to consequences of Friendliness - for example, the
> that Old Earth needs to be kept free of coercive transhuman memes, and
> thus the idea that transhumans and humans shouldn't interact.
My question is: What boundaries can we expect to be formed when
goes through the roof? I'm sure this is a question to be answered by an
or Sysop, but maybe someone would venture some interesting speculation
Imagine transhuman Bill becomes vastly more intelligent than person
much more that Bill can easily influence Alan to do things Alan might
to do on his own.
Fortunately, the Sysop is there to protect Alan by advising against
which might have deleterious outcomes for him.
Now, imagine Charlie. Transhuman Charlie is sufficiently more
transhuman Bill. Again our favorite Sysop can warn Bill off from doing
thing he might not want to do, if only he knew all the consequences. But
is a new twist with this case. Transhuman Charlie might convince
to modify vis own volition so that accepting new memes from Charlie is
desired. Transhuman Charlie might advertise his suggested memes cleverly
transhuman Bill bill chooses them with no violation of vis volition.
The result? Exponential cults win (in a weird way).
Cults do not violate volition, do they?
Will there be protections instead? Classifications of Transhuman
Level N may not try to influence anyone below Level N....
These seem to me to be the interesting kinds of problems that arise once
chosen volition as the "be all, end all" metric of the universe (and
currently subscribe to this view).
-- Durant Schoon
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:36 MDT