From: Chris Capel (pdf23ds@gmail.com)
Date: Sat Jul 23 2005 - 15:15:41 MDT
On 7/23/05, Russell Wallace <russell.wallace@gmail.com> wrote:
> On 7/23/05, Eliezer S. Yudkowsky <sentience@pobox.com> wrote:
> > The problem here is the ratio of cubic expansion through galaxies to
> > exponential reproduction. CEV doesn't solve this problem of itself, though it
> > might search out a solution. Neither does CEV create the problem or make it
> > any worse. Why do you suppose that we want to lock ourselves into a little
> > box rather than expanding cubically?
>
> By "locked into a box" I don't mean a bounded volume of physical space
> - that's not the issue. I mean that everyone will be forced at all
> times to follow the will of the Collective in every deed, word and
> thought, with no escape, no refuge and no privacy; it doesn't matter
> whether their material substrate is a meter or a gigameter away from
> the center.
Can you explain this a bit more? There's a specific scenario I have in
mind. The AI is created to determine the CEV of humanity, and it
decides that the CEV is to make a large number of different societies
run on different principles, some of which are insulated from the
others. Do you admit that this scenario is possible? Do you think that
CV inevitably leads to a single society in which certain overbearing
moral principles are absolute? Because it seems to me that it's
extremely unlikely for a non-broken CV AI to decide that humanity's
CEV is that every thought, word, and deed be controlled according to
this or that moral system. Unless you can show that this is an
inherent property in CV (what you seem to be arguing) or that it's
somehow the most likely CEV of humanity as you see it, then this rings
hollow to me. Does the very existence of an AI who has the ability to
determine the fate of every individual necessary entail that not one
thought, word, and deed is any longer free?
Chris Capel
-- "What is it like to be a bat? What is it like to bat a bee? What is it like to be a bee being batted? What is it like to be a batted bee?" -- The Mind's I (Hofstadter, Dennet)
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT