Re: CEV specifies who the AI cares about (was Re: Can't afford to rescue cows)

From: Stefan Pernar (
Date: Mon Apr 28 2008 - 06:21:19 MDT

On Sun, Apr 27, 2008 at 11:21 AM, Matt Mahoney <> wrote:

> --- Stefan Pernar <> wrote:
> > I was following the discussion from the sideline but think it is time
> > to point you to an alternative to Elizier's CEV.
> >
> > You can find my paper on friendliness called 'Practical Benevolence -
> > a Rational Philosophy of Morality' at:
> >
> >
> >
> > In it I combine Kantian moral philosophy with Darwinian evolution to
> > form a moral theory based in rational choice.
> In section 1.7 you state that existence is preferable to non-existence,
> and that existence is defined as the ability to be perceived. From
> this you conclude that it is the agent's interest to ensure continuous
> co-existence.
> This is the crux of your argument. I believe it is flawed. Evolution
> does not define existence as the ability to be perceived. It only
> matters that your genes are propagated. It does not matter if other
> animals are aware of your existence for you to exist. If they are
> predators or prey, it is preferable that they do not perceive you.

Very good point. I will have to meditate over this, however I do not see it
as invalidating the underlying logic = 'to exist is preferable over not to
exist' and the derived conclusions. Let me redefine it better. Maybe 'to
exist = being part of reality'? Suggestions?

Humans have a goal of being perceived by other humans. This is called
> "ego". Tribes whose members desire attention have a competitive
> advantage because ego leads to sharing of information, a common
> language, reciprocal attention and friendship, and ability to organize
> into companies and armies. For example, we would not post
> noncommercial messages to the internet without ego.
> But ego is not universal. Many animals are nonsocial. Some social
> species like ants and bees act out of instinct rather than learn social
> behavior out of a desire for attention.
> -- Matt Mahoney,

Totally agree.

Stefan Pernar
3-E-101 Silver Maple Garden
#6 Cai Hong Road, Da Shan Zi
Chao Yang District
100015 Beijing
Mobil: +86 1391 009 1931
Skype: Stefan.Pernar

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:02 MDT