From: Jef Allbright (jef@jefallbright.net)
Date: Wed Jun 07 2006 - 10:28:22 MDT
On 6/6/06, Eliezer S. Yudkowsky <sentience@pobox.com> wrote:
> I espouse the
> Proactionary Principle for everything *except* existential risks.
The Proactionary Principle is a putative optimum strategy for progress
within an inherently risky and uncertain environment. How do you
reconcile your belief in abandoning a strategy, not because the
problem definition has changed, but because your feelings about the
risk change past some threshold of problem scope?
> The mistake I made in 1996, when I thought any SI would automatically be
> Friendly, was a very bad mistake because it meant I didn't have to
> devote further thought to the problem. The mistake I made in 2001 was
> just as much a mistake, but it was a mistake that got me to spend much
> more time thinking about the problem and study related fields and hold
> myself to a higher standard.
I think you'll eventually reach a higher level of understanding and
leave behind the noble but unattainable dream of saving humanity, and
will refocus your considerable talents toward the even nobler and
realistic dream of maximizing the growth of human values that work
over increasing scope, while acknowledging that there will always be
risk.
> So,
> although it sounds absurd, you *should* be able to know in advance what
> you can and can't predict, and test those predictions you dare make, and
> use that same strict standard to predict the AI will be Friendly. You
> should be able to win in that way if you can win at all, which is the
> point of the requirement.
Your writing indicates that you're aware that you are presuming your
desired conclusion. You have begun to qualify your statements with
"if you can win at all", and that's a crucial "if". Maybe it's time
to go back and take another look at the economics of Pascal's Wager.
While I understand you appreciate direct criticism -- Crocker's Rules
and all -- I do also want to say that I support your efforts, have
always admired your integrity, and am increasingly impressed with the
quality of your public presentations. You're championing an important
line of thought.
- Jef
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT