From: Robin Lee Powell (rlpowell@digitalkingdom.org)
Date: Tue Nov 09 2010 - 13:06:57 MST
On Tue, Nov 09, 2010 at 10:07:20PM +0300, Alexei Turchin wrote:
[snip]
> 2) the AI should obey commands of its creator, and clearly understand who is
> the creator and what is the format of commands.
>
> 3) AI must comply with all existing CRIMINAL an CIVIL laws. [snip]
>
> 4) the AI should not have secrets from their creator. Moreover, he is
> obliged to inform him of all his thoughts. This avoids rebel of AI.
A more useful comment/response: the fact that what you just said
sounds easier to you than "the AI should be nice to people" is *an
illusion*. It's an artifact of your having a very complex brain
optimized for certain social interactions; you imagine that all
other minds will have similar complexity.
An example/exercise:
Try to seriously think about *EVERYTHING* you need to know to go
from "I need to get this piano out of my second story apartment, and
I don't care what happens to it" to "I should look down carefully
before I throw the thing out the window".
Some of these things:
* humans are fragile and hard to repair
** what "repair" means
** why a human would want to repair injury
** why other humans would want to repair injury to an innocent injured bystander
** what "innocent" means: why certain injuries are morally justified and others aren't
* humans care about being hurt
** what "caring" means
** understanding that others have values that might need to be respected
** why humans care about being hurt: what pain means, what death means
* a falling piano will do severe damage to a human
** what "falling" means
** how gravity works
** how mass and speed interact to create force
* humans are likely to be in the street below, and even if it isn't likely the risk of severe injury mandates that I check
There are many, many more; that's off the top of my head.
If you dry to drill to first principles on *every* aspect of that,
it's a staggering amount of information.
And not just information, *knowledge*. And wisdom, really.
Because not only do you need to know all these things, they all need
to connect to each other. "I need to get rid of this piano" needs
to lead to "this apartment has a window" to "I can throw things out
of the window" to "I wonder if anyone's down there?" to "this might
hurt somebody, I'd better check".
This is a *STAGGERING* amount of knowledge and wisdom. Now multiply
that by everything that has to be understood to *obey any possible
command*, let alone obey even a significant fraction of any
significant legal code.
If it can maintain all of that through recursive self improvement,
the AI is so wise you might as well just say "be nice to humans" and
call it good; you've already done all the hard work.
-Robin
-- http://intelligence.org/ : Our last, best hope for a fantastic future. Lojban (http://www.lojban.org/): The language in which "this parrot is dead" is "ti poi spitaki cu morsi", but "this sentence is false" is "na nei". My personal page: http://www.digitalkingdom.org/rlp/
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:05 MDT