Re: Volitional Morality and Action Judgement

From: Marc Geddes (marc_geddes@yahoo.co.nz)
Date: Mon May 24 2004 - 00:49:50 MDT


 --- Eliezer Yudkowsky <sentience@pobox.com> wrote: >
>
> The C-word ("consciousness"). The Q-word
> ("qualia"). That which causes
> us to mistakenly believe that if we think, therefore
> we must exist.
> That which our future selves shall come to define as
> personhood. What I
> want to say is just, "I don't want to hurt a
> person", but I don't know
> what a person is. If I could give a more specific
> definition, I would
> have already solved the problem.
>
> I need to figure out how to make a generic
> reflective Bayesian reasoner
> with a flaw in its cognitive architecture that
> causes it to be puzzled
> by the certainty of its own existence, and ask
> nonsensical questions
> such as "Why does anything exist?". Then I'll know
> what *not* to do.
>
> It worries me that our future selves may come to
> define personhood by
> reference to other qualities than the C-word, but
> there has to be
> somewhere to draw the line. Natural selection isn't
> an entity I
> sympathize with, and yet natural selection is a
> functioning, if
> ridiculously inefficient, optimization process. I
> figure that if I find
> a structure that I can exclude to definitely rule
> out the C-word, and if
> I also use pure Bayesian decision theory in place of
> pleasure and pain,
> that'll cover most of the bases.
>
> --
> Eliezer S. Yudkowsky
> http://intelligence.org/
> Research Fellow, Singularity Institute for
> Artificial Intelligence

Eli,

I respectfully continue to point out that I think that
your approach to FAI is the equivalent of trying to
'square the circle'.

*General intelligence without consciousness?

Impossible

*General intelligence that isn't observer centered?

Impossible

*General intelligence without a self?

Impossible

*Total altruism?

Impossible

*FAI that ends up with a morality totally independent
of the programmers?

Impossible

*FAI dedicated to 'saving the world'

Impossible AND mis-guided

More likely any general intelligence necessarily has
have: a 'self', consciousness, some degree of
observer centeredness, some non-altrustic aspects to
its morality, some input from the 'personal' level
into its morality, and helping the world would only be
a *secondary* consequence of it's main goals.

Of course, all this is only my opinion. Take it or
leave it. But don't say I didn't tell you so...

=====
"Live Free or Die, Death is not the Worst of Evils."
                                      - Gen. John Stark

"The Universe...or nothing!"
                                      -H.G.Wells

Please visit my web-sites.

Science-Fiction and Fantasy: http://www.prometheuscrack.com
Science, A.I, Maths : http://www.riemannai.org

Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com



This archive was generated by hypermail 2.1.5 : Tue Feb 21 2006 - 04:22:36 MST