From: Smigrodzki, Rafal (SmigrodzkiR@msx.upmc.edu)
Date: Thu Jun 27 2002 - 15:50:35 MDT
Eliezer S. Yudkowsky [mailto:sentience@pobox.com] wrote:
Killing people for what seem like good reasons might tend
to corrupt a human, but to the best my knowledge, I can't see it affecting a
Friendly AI.
### The AI will have a very complex network of hierarchical goals, as well
as Bayesian priors, being adjusted to function in a complex environment with
uncertain information. Don't you think that there might be a pervasive
change, an accumulation of small biases towards violent solutions? Whenever
the AI has to make a decision using uncertain data, the verification of the
validity of the decision might take a long time, and might be affected by
the current state of the AI. Once in a state predisposing to violent
behaviors, self-reinforcing patterns could emerge, with violence leading to
violent responses, and the (perceived) need for more violent responses,
especially if there are multiple AI's involved. Friendliness is difficult if
there is no possiblity of verifying the motives and actions of game
participants, and this would be the case with warring AI's. Their
motivations would be opaque to each other, therefore subject to the same
type of social dynamics that occurs in humans incapable of monitoring each
other's behavior (first strikes, arms races, etc.) The FAI might learn the
meaning of being Friendly to humans but I think it would have difficulty
with learning to be friendly to the combination of tribal humans with their
own tribal-Friendly AI's. As you later say, the growing up of the infant AI
might be unfavorably affected, in the distant analogy to the detrimental
effects of early childhood emotional trauma in humans.
Rafal
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:39 MDT