From: Mark Waser (mwaser@cox.net)
Date: Sun Mar 16 2008 - 19:50:32 MDT
Hi Peter,
Unfortunately, you didn't give me the all of the information necessary
(i.e. my assessment of whose beliefs are more likely to be correct).
If I believe that Alf's and Beth's beliefs were individually each as
likely to be as correct as mine, then I would want Beth next to the bomb.
If I have reason to believe that my beliefs were more likely (by
Bayesian propagation) to be correct despite the fact that there are two of
them and only one of me, then I would want Alf next to the bomb.
Also note that I am assuming that you mean to have the condition that I
*must* have one of them next to the bomb as part of the problem (otherwise,
the correct answer is "I want someone who will just leave the bomb alone and
not do anything -- unless it is visibly ticking down and near zero).
But currently, I have the belief that I don't understand how this
relates to my solution.
----- Original Message -----
From: "Peter de Blanc" <peter@spaceandgames.com>
To: <sl4@sl4.org>
Sent: Sunday, March 16, 2008 11:14 AM
Subject: Re: Friendliness SOLVED!
> On Wed, 2008-03-12 at 23:56 -0400, Thomas McCabe wrote:
>> This is how humans usually act; it is not how most AIs will act. Iterm
>> #1,782 on my agenda is to prove that, except for special cases, you
>> get a higher expected utility when another agent shares your utility
>> function than when the two agents have different utility functions.
>> Hence, forced modification of the other agent's utility function also
>> has positive utility.
>
> The "special cases" are not as special as you may think. I have one for
> you. There's a bomb in some populated area. You believe that cutting the
> red wire will detonate it, and cutting the blue wire will disarm it.
>
> So, to summarize your beliefs:
>
> Red -> detonate
> Blue -> disarm
>
> Alf and Beth both believe:
>
> Red -> disarm
> Blue -> detonate
>
> Alf wants to detonate the bomb, and Beth wants to disarm it. Which one
> would you rather have standing next to the bomb with wire-cutters?
>
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:02 MDT