From: Johnicholas Hines (johnicholas.hines@gmail.com)
Date: Mon Feb 09 2009 - 16:37:20 MST
On Mon, Feb 9, 2009 at 4:29 PM, Matt Mahoney <matmahoney@yahoo.com> wrote:
> --- On Mon, 2/9/09, Vladimir Nesov <robotact@gmail.com> wrote:
>> On Mon, Feb 9, 2009 at 9:26 PM, Petter Wingren-Rasmussen
>> <petterwr@gmail.com> wrote:
>> > For me the conclusion is clear:
>> > Any AI with a hard-coded law for "being nice","prevent murders" or any
>> > other non-modifiable procedure will with great probability be overrun
>> > by a system without those hardwritten rules.
>> >
>>
>> This is vacuous: is it bad that civilization got destroyed, because
>> there was no moral way of avoiding destruction? From that
>> civilization's perspective, it just might be an optimal course of
>> action, it won even though it's destroyed.
>
> Morals are not weak or strong, only different. Whatever happens to the human race will be judged good from the perspective of the species that replaces us.
>
> -- Matt Mahoney, matmahoney@yahoo.com
>
Would the "positive/normative" distinction be valuable here? I think
it's fairly standard in philosophy and economics. As I understand it,
statements are called "positive" if they are about what is the case
(or what was the case, or what will be the case). Statements are
called "normative" if they are about whether a situation is desirable
or undesirable; what ought to be.
If I'm parsing the various speakers correctly, Petter
Wingren-Rasmussen made a positive statement something like: "Any such
AI will suffer such and so."
Vladimir Nesov misunderstood Petter's prediction as a normative
statement something like "We should strive to build amoral AIs." and
responds with a rhetorical question. His question strongly implies a
normative statement, something like: "We should strive to behave well,
even if it means our destruction."
Then Matt Mahoney responds with a strictly positive statement. In my
experience, Dr. Mahoney strives to only use positive statements, never
normative ones - I'm not sure why.
Hope this helps,
Johnicholas
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:04 MDT