From: Bill Hibbard (test@demedici.ssec.wisc.edu)
Date: Sat May 17 2003 - 15:08:31 MDT
On Fri, 16 May 2003, Eliezer S. Yudkowsky wrote:
> Bill Hibbard wrote:
> >
> > The ambiguous definitions in the SIAI analysis will be
> > exploited by powerful people and institutions to create
> > AIs that protect and enhance their own interests.
>
> I am not going to say that the problem of third-party enforcement of
> Friendliness, given a programmer actively trying to twist the regulations,
> is "un-solvable". I have learned better than to use words like that. The
> word "impossible" means "I do not know how to do this right now; ask me
> again tomorrow."
>
> Bearing that in mind: the problem you describe is impossible.
>
> I will be ecstatic if the problem of Friendliness is solvable by someone
> who genuinely wants to solve it. The problem given a hostile programmer
> is... how can I put this... INSANE. If you want technological advancement
> on the problem of trust, build a better lie detector using advances in
> neuroimaging, signal processing, and pattern recognition, and run the
> programmers through it. I hereby volunteer
This is an interesting issue, that I address in the "Outlaws"
section of my book, in the "Public Education and Control"
chapter. It will be impossible for regulation to prevent all
construction of unsafe AIs, just as it is impossible to prevent
all crimes of any kind. But for an unsafe AI to pose a real
threat it must have power in the world, meaning either control
over significant weapons (including things like 767s), or access
to significant numbers of humans. But having such power in the
world will make the AI detectable, so that it can be inspected
to determine whether it conforms to safety regulations.
I don't think that determining the state of mind of the
programmers who build AIs is all that relevant, just as the
state of mind of nuclear engineers isn't that relevant to
safety inspections of nuclear power plants. The focus belongs
on the artifact itself.
The danger of outlaws will increase as the technology for
intelligent artifacts becomes easier. But as time passes we
will also have the help of safe AIs to help detect and
inspect other AIs.
----------------------------------------------------------
Bill Hibbard, SSEC, 1225 W. Dayton St., Madison, WI 53706
test@demedici.ssec.wisc.edu 608-263-4427 fax: 608-263-6738
http://www.ssec.wisc.edu/~billh/vis.html
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:42 MDT