Re: SIAI's flawed friendliness analysis

From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Wed May 21 2003 - 14:38:13 MDT


Robin Lee Powell wrote:
> On Wed, May 21, 2003 at 04:33:25PM -0300, Ben Goertzel wrote:
>
>>Secondly, even if it DID give the smartest researchers an
>>advantage, why do you assume that smarter researchers are more
>>likely to succeed at creating FAI?
>
> Given the incidence of mental problems in *really* smart people,
> especially Asperger's, I'd suggest that a nearly inverse relation
> holds true.

Friendliness is not communicated to AIs by having it sneezed onto them by
loving researchers. It is a terrifying and counterintuitive truth that
kindness is not automagically contagious across species. Getting morality
from one place to another requires a rather large amount of intelligence.
  Failing that, no amount of good intentions will help. Good intentions
are not universal, but they are far easier to find than the necessary
level of intelligence.

-- 
Eliezer S. Yudkowsky                          http://intelligence.org/
Research Fellow, Singularity Institute for Artificial Intelligence


This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:42 MDT