From: Robin Lee Powell (firstname.lastname@example.org)
Date: Wed Oct 07 2009 - 10:13:22 MDT
On Wed, Oct 07, 2009 at 01:26:15PM +0000, Randall Randall wrote:
> On Wed, Oct 07, 2009 at 08:41:01AM +0100, Stuart Armstrong wrote:
> > >> If you saw a random baby lying on the sidewalk, you would not
> > >> kill it. ?This is a "limitation" in the human architecture.
> > >> ?Do you find yourself fighting against this built-in
> > >> limitation? ?Do you find yourself thinking, "You know, my
> > >> life would be so much better if I wanted to kill babies.
> > >
> > > If you substituted the word "baby" for "slug" you would have a
> > > much more realistic analogy;
> > Um - no you wouldn't. You'd get an massively less realistic
> > analogy; slugs are things we hate and value not at all. The
> > process analogised is going from valuing something very highly
> > to valuing something much less; loving babies but voluntarily
> > deciding to treat babies as slugs.
> Of course, John is talking about the intelligence difference,
> which he sees as overriding all that "goals" business.
Yes. This is so blatantly insane, and he seems to not actually
absorb anything anyone says on the topic, that I wasn't really
talking to him. I just wanted to make sure it didn't go
unchallenged, since there seem to be newbies around.
> Some people do love and highly value their houseplants, which
> might be an analogy you can both agree on.
Very, very few people would run into a burning house to save their
houseplants; that's just not a strong enough emotional attachment to
be a decent analogy. I guess that's sort of the boundary for me:
take something you care enough about that you would run into a
burning house to save it; do you feel "restrained" by the fact that
you can't want to kill that thing for fun? Do you wish to fix that
The entire idea is preposterous. Believing that such a thing would
occur shows an utter lack of understanding of the entire concept of
goals and/or utility functions. I'd say it shows an utter lack of
understanding of the entire concept of *intelligence*, but no-one
understands intelligence well enough to make a claim like that, I
-- They say: "The first AIs will be built by the military as weapons." And I'm thinking: "Does it even occur to you to try for something other than the default outcome?" See http://shrunklink.com/cdiz http://www.digitalkingdom.org/~rlpowell/ *** http://www.lojban.org/
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:04 MDT