Re: How hard a Singularity?

From: James Higgins (
Date: Wed Jun 26 2002 - 00:45:10 MDT

At 10:21 PM 6/25/2002 -0700, you wrote:
>At 08:24 PM 6/25/2002 -0700, you wrote:
>>>Let's say you're 90% (+-5%) sure that the answer is: 1000. Would you
>>>still take the time to consult with someone else? With ten
>>>others? What if 1 out of the 10 you consult says the answer is
>>>990? Would you still get 20 more?*
>>>It is rational to check with others to see if your theories of
>>>Friendliness, but it doesn't make sense to not move forward just because
>>>someone disagrees.
>>Wrong, dead wrong. For one thing, why bother to check with others if you
>>just plan to ignore them anyway? It is irrational to proceed unless
>>there is no other choice. ABSOLUTELY ANYTHING that could lead to the
>>extinction of the human race MUST BE checked, rechecked, checked again,
>>etc. To the maximum extent possible. A 10% risk of extinction is way,
>>way too high. Unless, of course, the result of not acting IMMEDIATELY
>>became equal to or worse than the result of failure. Maybe at 0.01%
>>chance of extinction I might back off my stance if a very significant
>>number of deaths were occurring continuously.
>>James Higgins
>Fair enough. I should have been more clear. When asking a large enough
>audience about a topic such as designed Friendliness, where we have no
>experience, we can expect people to disagree. We can't absolutely reject
>a plan just because someone disagrees, they have to have a valid
>(checkable) reason for disagreeing.

True, there will always be some descent. I don't expect, and would not
require, 100% agreement to proceed. However, with something as significant
and dangerous as the Singularity the descent must be rigorously and
thoroughly examined as much as possible. Also, just because an issue
raised may not be checkable doesn't mean it isn't valid. I don't see how
the whole friendliness system proposed is checkable. Heck, I don't even
see how it is workable in most cases. The idea that it could remain
essentially unchanged after many thousands, quite possibly millions, of
subjective modifications, most likely including complete rewrites, seems
rather far fetched. At least without some serious proof.

James Higgins

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:39 MDT