From: Elias Sinderson (elias@cse.ucsc.edu)
Date: Tue Feb 22 2005 - 18:38:52 MST
Ben Goertzel wrote:
> ITSSIM would cause an AI to avert only existential risks that either
> -- it could avert without compromising its principle of safe action, or
> -- it judged would be likely to mangle ITSELF, therefore potentially
> threatening its ability to act safely
> I don't think that ITSSIM is an ideal approach by any means, in fact
> it's over-conservative for my taste. I'd rather have an AI be able to
> deviate from its safe-self-modification rule in order to save the
> universe from other threats; the question is how to allow this without
> opening the door to all sorts of distortions and delusions...
I should think that an approach similar to pushing / popping a stack
frame would be useful - pushing a new operational state and popping it
when a perceived threat had been mitigated in order to return to the old
state. This presumes that the self-modifying aspect of the codebase was
sufficiently isolated from being modified, itself, so as to ensure that
the push / pop strategy wasn't affected. This, of course, raises a
number of other questions related to recursive frames of reference,
representation of 'self', etc.
Regards,
Elias
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:50 MDT