From: Amon Zero (amon@doctrinezero.com)
Date: Sun Mar 06 2011 - 06:14:23 MST
Hi All -
I'm not sure if this list even exists anymore, but just in case it does... I've
been thinking about AGI and Friendliness. Yes, I know, a minefield to say
the least. Specifically, I've been taking this matter and comparing it to
early Extropian notions about libertarianism and technological progress, and
the comparison suggests what might be a new question. (Something that I
daresay Ben Goertzel has considered, but I don't have him to hand, as it
were).
So, I remember a piece of Max's (IIRC), in which he made the case that too
many governmental controls on technological development would only ensure
that less-controlled countries would develop key technologies first. Within
reason, that sounds a plausible claim to me. Universally Friendly AGI, of
the sort that SIAI contemplates, seems to be a textbook case of constrained
technological development. i.e. it seems reasonable to expect that
non-Friendly AGI would be easier to develop than Friendly AGI (even if FAI
is possible, and there seem to be good reasons to believe that universally
Friendly superhuman AGI would be impossible for humans to develop).
Because Friendliness is being worked on for very good (safety) reasons, it
seems to me that we should be thinking about the possibility of "locally
Friendly" AGI, just in case Friendliness is in principle possible, but the
full package SIAI hopes for would just come along too late to be useful.
By "locally Friendly", I mean an AGI that respects certain boundaries, is
Friendly to *certain* people and principles, but not ALL of them. E.g. a
"patriotic American" AGI. That may sound bad, but if you've got a choice
between that and a completely unconstrained AGI, at least the former would
protect the people it was developed by/for.
Anyway, before I go too far down this road, does anyone have any thoughts on
this idea? Maybe you know of a relevant article out there, someone who has
already considered the matter at length?
Cheers,
Amon
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:05 MDT