From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Mon Feb 04 2002 - 15:48:26 MST
Mitch Howe wrote:
>
> Maybe we disagree on what the purpose of an FAQ should be. If my FAQ was
> supposed to be taken as an attempt to convince everyone that a Sysop is a
> wonderful idea, I have failed miserably.
Hm. I think your document is much better structured than Gordon's (sorry,
Gordon!) so mostly the objections don't apply as much to yours. However,
I still remember one or two cases where I thought the explanation was too
sparse. Specifically:
"A Sysop is, by definition, the least intrusive system that can protect
the freedoms of all within its care and, by extension, secure the future
of humanity."
I worry that most people are going to read this and think "Prove it". The
problem is that "by definition" is often used in senses other than the
rationally proper one, so to send the right signal takes a stronger
phrasing, like: "A Sysop is supposed to be the least intrusive system
that can protect the freedoms of all within its care and, by extension,
secure the future of humanity."
> I agree that it wouldn't hurt to beef up some of the answers by providing
> more metaphors and links. I intend to do this. But I think it would be a
> mistake to project the idea that CFAI proves a Sysop to be the very
> embodiment of Friendliness. I didn't think this was your position; If it
> is, I confess to having missed the whole point of CFAI.
Actually, I was contrasting your version to Gordon's (sorry, Gordon!).
The problem is that it's not quite clear that your document is meant to be
descriptive rather than persuasive. It's a trick of the language, I
think. When I look at a Q like "Why would we need a special system to
ensure our individual and collective protection?" it seems like the Q is
being asked by an individual who is skeptical of the Sysop Scenario,
rather than a person who wants to know more about what the Sysop Scenario
is; therefore, when I read the answer, I'm sort of looking at it as being
of the form "This is why you need this special system", not "This is why
some people think you need this special system." The ubiquitous use of
"We wouldn't, necessarily" is a good start but doesn't quite dispel the
expectation. What it really needs is an agnostic phrasing for the
questions, or an intro where it explicitly says "I am not for or against
the Sysop scenario, just trying to describe it."
-- -- -- -- --
Eliezer S. Yudkowsky http://intelligence.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:37 MDT