From: Norman Noman (overturnedchair@gmail.com)
Date: Fri Oct 26 2007 - 00:08:55 MDT
On 10/25/07, Robin Brandt <mandelum@gmail.com> wrote:
> The most important thing being to replace my Darwinian supergoal with
> a more high level ethical goal, consistent with my worldview, which is
> of course somehow the product of my supergoal, but without having
> replaced it.
So, you want to replace your darwinian supergoal without replacing it. You
want I should check your blinker light fluid while I'm at it?
Folks, there is no "darwinian supergoal" anywhere in your brain. It's true
that we're evolved to eat because eating > survival > reproduction > more
copies of the old genetic code, but this is not part of our actual human
volition to any significant degree. Eating is programmed in us as an end
unto itself. When an amoeba moves towards a heat source, it's not thinking
"this will improve my chances of reproduction". It's not even thinking that
SUBCONSCIOUSLY. And neither are we, when we move toward heat sources.
A person's supergoal (or utility function, i believe is now the more
accepted term), to the extent that it can be defined at all, is a constantly
changing, incoherent, paradoxically self-referential mess. What you would
get if you let it re-write itself depends to an enormous extent on the
details of the proceedure, and the specific state of mind you started with.
Give a curious person the tools and they'll become a knowledge-eater, give
an angry person the tools and they'll become an engine of destruction, give
a paranoid person the tools and they'll become a paranoid person with no
remaining desire to change in any way.
I see no "fair" way to iron this out. It's a chaotic system. If you let it
out, you get chaos.
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:58 MDT