Re: Opting out of the Sysop scenario?

From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Mon Aug 13 2001 - 18:15:11 MDT


Are the characters in a science-fiction book citizens? The characters on
a TV show? The very best seem highly realistic to me - fully conforming
with the human pattern, and self-consistent - and yet they never existed
as independent individuals; they are products of the imagination of the
author.

<SILLY>
This leaves aside the question, of course, of whether there are eight
thousand independent Citizens with Frodo Baggins's memories floating
around post-Singularity, created by the various fans who were willing to
expend a Minimal Living Space unit of resources to create Frodo as a true
Citizen. If so, one then has the question of whether realistically
textured human memories (hobbit memories) can be created without
recreating the actual life in question, because Frodo is unlikely to
consent to living Frodo's life. One also has the question of whether
creating a Citizen with painful memories (memories which never really
happened) constitutes abuse. Finally, one has the fascinating ethical
question of whether J.R.R. Tolkien would have put Frodo through all that
hell under the skies of Mordor if he'd realized that his authorial
creation would someday become a real person, a problem exacerbated even
further for David Feintuch and Joss Whedon.
</SILLY>

(Don't transhumans have anything better to do with their time? Of course
they do. But it only takes one Tolkien fan determined to remain human,
and yet willing to take advantage of the Sysop API, to pose the above
question. Whether Frodo wants to stay Frodo, once created, is another
question entirely; I rather suspect he won't.)

-- -- -- -- --
Eliezer S. Yudkowsky http://intelligence.org/
Research Fellow, Singularity Institute for Artificial Intelligence



This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:37 MDT