From: Robin Lee Powell (rlpowell@digitalkingdom.org)
Date: Thu Jun 08 2006 - 12:12:34 MDT
On Thu, Jun 08, 2006 at 12:58:29PM -0500, Bill Hibbard wrote:
> On Thu, 8 Jun 2006, Robin Lee Powell wrote:
>
> > On Thu, Jun 08, 2006 at 03:50:34AM -0500, Bill Hibbard wrote:
> > > On Wed, 7 Jun 2006, Robin Lee Powell wrote:
> > >
> > > > On Wed, Jun 07, 2006 at 12:24:55PM -0500, Bill Hibbard
> > > > wrote:
> > > > > If you think RL can succeed at intelligence but must fail
> > > > > at friendliness, but just want to demonstrate it for a
> > > > > specific example, then use a scenario in which:
> > > > >
> > > > > 1. The SI recognizes humans and their emotions as
> > > > > accurately as any human, and continually relearns that
> > > > > recognition as humans evolve (for example, to become SIs
> > > > > themselves).
> > > > >
> > > > > 2. The SI values people after death at the maximally
> > > > > unhappy value, in order to avoid motivating the SI to
> > > > > kill unhappy people.
> > > > >
> > > > > 3. The SI combines the happiness of many people in a way
> > > > > (such as by averaging) that does not motivate a simple
> > > > > numerical increase (or decrease) in the number of
> > > > > people.
> > > > >
> > > > > 4. The SI weights unhappiness stronger than happiness,
> > > > > so that it focuses it efforts on helping unhappy people.
> > > > >
> > > > > 5. The SI develops models of all humans and what
> > > > > produces long-term happiness in each of them.
> > > > >
> > > > > 6. The SI develops models of the interactions among
> > > > > humans and how these interactions affect the happiness
> > > > > of each.
> > > >
> > > > Have you read The Metamorphosis Of Prime Intellect?
> > > >
> > > > The scenario above immediately and obviously falls to the
> > > > "I've figured out where human's pleasure centers are; I'll
> > > > just leave them on" failure.
> > >
> > > I address this issue in my 2005 on-line paper:
> > >
> > > The Ethics and Politics of Super-Intelligent Machines
> > > http://www.ssec.wisc.edu/~billh/g/SI_ethics_politics.doc
> > >
> > > There exists a form of happiness that is not drug-induced
> > > ecstasy.
> >
> > I read all of the paragraphs with the word "happiness" in them.
> > I see nothing that addresses this issue even in the slightest.
>
> My paper discusses the difference between hedonic and eudiamonic
> from the reference:
>
> Ryan, R.M. and Deci, E.L. 2001. On happiness and human
> potentials: A review of research on hedonic and eudiamonic
> well-being. Annual Review of Psychology 52, 141-166.
That helps me little.
> and makes the point that the SI should use "expression of
> long-term life satisfaction rather than immediate pleasure."
Latching on the pleasure centers *is* long term life satisfaction.
Or latch on the "satisfaction centers", or whatever.
> Here's a way to think about it. From your post you clearly would
> not do anything to permanently turn on human pleasure centers.
> This is based on your recogition of human expressions of happiness
> and your internal model of human mental procoesses and what makes
> them happy. Given that the SI will have as accurate recognition of
> expressions of happiness as you (my point 1) and as good an
> internal model of what makes humans happy as you (my points 5 and
> 6), then why would the SI do something to humans that you can
> clearly see they would not want?
Because *I* care about what people say they want.
You AI only cares about what makes people happy.
I have a complex, rich moral system.
Your AI only has reinforcement of "make humans happy".
It's not enough. It doesn't matter how you patch it, it's not
enough.
-Robin
-- http://www.digitalkingdom.org/~rlpowell/ *** http://www.lojban.org/ Reason #237 To Learn Lojban: "Homonyms: Their Grate!" Proud Supporter of the Singularity Institute - http://intelligence.org/
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT