From: Norm Wilson (web64486@programmar.com)
Date: Tue Jun 01 2004 - 11:12:03 MDT
Eliezer Yudkowsky wrote:
> Self-determination is not the only criterion of an
> acceptable outcome.
Is self-determination a criterion of an acceptable outcome? Sounds like speculation about the outcome to me :)
> It seems to me that an FAI can make huge improvements to
> background rules before that starts interfering with
> self-determination.
Who are we to assume that an FAI will value our self-determination at all, unless we make it an invariant?
For that matter, the idea that collective volition is a correct path to morality is arguably itself a moral judgment. By making it an invariant, you run the risk of imposing moral content on the FAI?s morality-seeking structure.
While I like the idea of specifying friendliness "structure" over its "content", I think it will be difficult to introduce invariants that don?t themselves have implicit moral content.
---------
Norm Wilson
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT