Re: [sl4] Long-term goals (was Re: ... Why It Wants Out)

From: Martin Sawitzki (blich@gmx.de)
Date: Mon Jun 30 2008 - 06:52:17 MDT


> Tim writes
> > By the way, if the AI has any long-term goals, then it will want to
> > preserve its own integrity in order to preserve those goals. Although
> > "preserve its own integrity" is a good enough example for the issue at
> > hand, it's not something you'd really need to put in there explicitly.

Strong sense of identity and self-modifying are somewhat contradictions. (similar to drugs that change the personality) That might even lead to the point that a self-aware AI does not want to modify itself to preserve its current state of being.

Likewise a friendly AI might be aware that the next-level AI it's supposed to develop might not be guaranteed friendly and therefore refuse to develop it at all.

Yours,

Martin

[obligatory new poster and foreign speaker disclaimer]

-- 
Ist Ihr Browser Vista-kompatibel? Jetzt die neuesten 
Browser-Versionen downloaden: http://www.gmx.net/de/go/browser
-- 
GMX startet ShortView.de. Hier findest Du Leute mit Deinen Interessen!
Jetzt dabei sein: http://www.shortview.de/wasistshortview.php?mc=sv_ext_mf@gmx


This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:03 MDT