Re: Geddes's 'Moral Perturbation Theory'

From: Eliezer Yudkowsky (
Date: Thu Jun 17 2004 - 22:24:34 MDT

Marc Geddes wrote:
> To approximate the collective volition of humanity we
> don't need to look at the brains of 6 billion people
> at all. In fact I think that the brains of only a few
> hundred should suffice as a very good approximation!


> Further if we wanted to calculate the approximate
> extrapolated volition of any particular individual,
> we'd simply take the CV as a generic first order
> approximation , then add the 'moral perturbations'
> caused by the more specific mental characteristics of
> the individual under consideration.

This doesn't follow. It's like saying that to compute the position of an
arbitrary solar object, you take the average position of the nine planets,
and then add the specific characteristics of the body in question. There
is a sense in which it is efficient to start with the Sun's extrapolated
position, to get the position of Asteroid #whatever in the galaxy and the
universe, and then add corrections. But this is not analogous to starting
with the collective volition as the generic first order approximation; it
is analogous to starting with the fact of your humanity (possibly,
masculinity or feminity) as the generic first order approximation. What
part of a collective volition is universal in individual volitions with
high probability (say, P > 95%) is a different problem from extrapolating a
collective volition as such.

PS: Geddes, can you please choose less excitable subject lines?

Eliezer S. Yudkowsky                
Research Fellow, Singularity Institute for Artificial Intelligence

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT