From: Ben Goertzel (ben@goertzel.org)
Date: Sun Feb 19 2006 - 22:39:11 MST
> Any external Friendliness Verifier will be more complex than the AI
> it is verifying (in which case, we are only shifting the problem) or
> it will not work. If the external FV has imperfect information, it
> can be gamed by exploiting the asymmetries in the Godel problem
> mentioned above.
But James, I was not proposing a full-fledged autonomous sentient AI
with an "external Friendliness verifier"....
I was proposing an AI with multiple internal components, potentially including
-- a hypothesis generator using evolutionary programming type methods,
which are speculative and include a quasirandom component
-- an inference module with friendliness verification built in
In this case, the hypothesis generator does not have its own flexible
goal system and is not configured to "game" anything nor will it
self-modify into something configured to "game" something. It is an
optimization machine that generates hypotheses satisfying criteria fed
to it by the inference module.
I agree that in this context the inference module will almost
certainly be more complex than the hypothesis generation module, in
terms of program length.
-- Ben G
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:55 MDT