From: Russell Wallace (russell.wallace@gmail.com)
Date: Tue May 10 2005 - 02:23:02 MDT
On 5/10/05, Sebastian Hagen <sebastian_hagen@gmx.de> wrote:
> The last judge is more of a final check to protect against unexpected
> failures; it shouldn't be necessary to avoid likely failure modes. Why
> do you think that this specific one is sufficiently likely to warrant
> specifically worrying about it?
Now I've cited several more that are also sufficiently likely to
warrant specifically worrying about, if that answers your question? :)
[1]
> Either 'sentient life' matters to humanity in general, in which case I'd
> expect the Extrapolated Collective Volition of humanity to ensure that
> it continues to exist (or fail harmlessly in case of nonconvergence,
> computational intractability, etc)
Or humanity cares about the continued existence of sentient life and
takes decisions it _thinks_ will ensure that; and the RPOP flashes a
picture on the screen that lets the Last Judge convince himself it
will do so; and it turns out otherwise.
> or it doesn't, in which case I don't
> have any particular reason to care if it is optimized away.
So you don't personally care if everything of value is destroyed?
> The conventional assumption in this case is that unupgraded humans, even
> with some predictive help from a RPOP, are not capable of reliably
> making good decisions about rules that are supposed to stay in force
> forever.
Conventional assumption? What convention would this be?
> This applies to the rules for individual domains, but it also
> applies to the whole idea of domain protection.
If you have a better idea, it's not too late.
> If there are critical differences in the values of different subsets of
> the population the CV still has the option implement an individual or
> domain protection system with permanently fixed rules; it would be in a
> much better position to make that decision than present-day humans.
[1] Is there an emoticon for humor of the "Nah, I was only having you
on, it's crucifixion really!" [2] variety?
[2] Monty Python, 'The Life of Brian'. (Do not die without having seen
this movie.)
- Russell
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT