Re: qualia-oriented SI (was: "friendly" humans?)

From: Wei Dai (weidai@weidai.com)
Date: Fri Jan 02 2004 - 20:23:50 MST


On Sat, Jan 03, 2004 at 03:34:56AM +0900, Metaqualia wrote:
> > This is an interesting idea, but won't such an SI just find the best
> > qualia possible and then fill the universe with copies of the structure
> > that most efficiently generates the optimal qualia, painlessly killing all
> > other life forms for their atoms?
>
> And what would you have it do instead?

I would like to have the SI explain to me, before having me killed, *why*
filling the universe with the optimal qualia is a worthwile thing to do
and what exactly makes that qualia better than all others, and perhaps
give me a chance for rebuttal. But I'm afraid such an SI would consider it
a waste of resources to keep me alive long enough to deliver the
explanation, and a security risk to allow me to talk back to it (a tiny
one to be sure, but still not negligible).

Maybe what you meant by the question is, what would I do instead if I was
an SI? Well, the first thing I would do is to think "really hard" about
what I should do. Beyond that... beats me. (The real me, that is,
hopefully the SI me would come up with something).



This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:43 MDT