Re: AGI Prototying Project

From: Russell Wallace (
Date: Tue Feb 22 2005 - 00:44:45 MST

On Tue, 22 Feb 2005 14:35:24 +1100, Tennessee Leeuwenburg
<> wrote:
> It remains unclear to me what "RPOP" is an acronym for; Google has no
> suggestions.

Really Powerful Optimization Process.

> Your argument :
> 1) If Friendliness is a fitter being, everything will be fine
> 2) If mindless nanotech is a fitter being, we're screwed
> 3) If (1) is true, then we are fine if we pursue Friendliness
> 4) If (2) is true, then we are screwed, but you maintain hope that we
> could work to not be screwed.

Actually, the drunk and the lamp post would be a better analogy for my
argument, which goes like this:

1) If nothing we can do can change the course of evolution, then
nothing we do matters in the long run; whether we're fine or we're
screwed, we can't do anything to influence our fate.
2) If there is something we can do to change the course of evolution,
then we can influence our fate.

Therefore, we should assume 2 is true, because if we're wrong it
doesn't matter anyway.

> It's a horizon thing. If we can get to friendliness FIRST, then
> evolution might not explore mindlessness.

Perhaps we were talking past each other then, because that's exactly
what I'm advocating; "circumventing evolution" is a term I'd use to
describe this, but if you want to use a different term then I don't
have a problem with that.

> It's not about 'escaping' evolutionary pressure - that is like saying
> that everything would be easier if the laws of the universe were
> different. Survival of the fittest /will/ happen. We need to ensure
> that something interesting is the fittest thing.

Okay, how do you propose to do this?

> Oh, and I read some of the mail archive, and ended up on a page by Ben
> Goetzel talking about "Solomonoff Induction". Is anyone interested in
> my pointing out some implied assumptions I found in there, or has this
> already been done to death?

I'd be interested in hearing them.

- Russell

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:50 MDT