From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Wed May 23 2001 - 17:34:12 MDT
Jimmy Wales wrote:
>
> "Respecting the wishes of others" can't be an ethical primary,
> as it leads directly to unanswerable questions -- the wishes of
> others will typically be in conflict.
I don't see why you think this makes volition-based Friendliness
impossible as an ethical primary. If you have a quantitative (or
comparative) measure of how well a local event violates or matches
"respecting the wishes of others", then that plus standard intelligence
allows for a good stab at global optimization.
Under volitional Friendliness, your volition only holds sway over the
things you own, including yourself. Making a coercive alteration to
someone else's property or self, no matter how much you want it, is not
moral. This renders volitional Friendliness a strictly local property
almost all of the time, at least as far as I can see. Not that optimizing
an intricate global property *would* present a problem for a transhuman;
we humans flinch away from that sort of thing just because our own
consciousnesses can only track a very small number of objects
simultaneously.
-- -- -- -- --
Eliezer S. Yudkowsky http://intelligence.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:36 MDT