From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Wed May 04 2005 - 21:52:52 MDT
Peter C. McCluskey wrote:
> tennessee@tennessee.id.au (Tennessee Leeuwenburg) writes:
>
>>Question : Do we have time to solve Friendliness before we are
>>confronted with the first real AIs?
>
> If CV is the right approach to Friendliness, then no. The cpu time
> needed to do CV right appears sufficient to delay development enough to
> make it irrelevant.
> Which is another reason to aim for making the first AI an obedient AI,
> even if we need Friendly AI soon after that.
From the PAQ list in _Collective Volition_:
**
Q10. How accurate and detailed does the extrapolation of collective volition
need to be in order to work? (Frequently Asked)
A10. Some important points that I don't think I emphasized clearly enough in
the body of the text:
* The optimization process is trying to extrapolate a collective
volition, not the volition of any one individual. I am visualizing and
describing this as an actual, detailed, attempt at extrapolating individual
minds and their social interaction; but that is because I am defining what I
wish to approximate.
* The optimization process is extrapolating a spread-out range of
possibilities, not the answer. Given the uncertainty of underlying quantum
processes, no definite answer may exist even in principle.
* Do we want our collective volition to satisfice, or maximize? My guess
is that we want our collective volition to satisfice - to apply emergency
first aid to human civilization, but not do humanity's work on our behalf, or
decide our futures for us. If so, rather than trying to guess the optimal
decision of a specific individual, the collective volition would pick a
solution that satisficed the spread of possibilities for the extrapolated
statistical aggregate of humankind.
This is another reason not to stand in awe of the judgments of an extrapolated
collective volition - a solution that satisfices an extrapolated spread of
possibilities for the statistical aggregate of humankind may not correspond to
the best decision of any individual, or even the best vote of any real, actual
adult humankind.
If all we want out of our collective volition is a limited set of emergency
assistances, measures on which our superposed possible volitions are
predictably mostly coherent, then the extrapolation doesn't need to be
extremely detailed. The extrapolation might be extremely detailed anyway - why
not? - but it wouldn't need to be detailed. Who knows, maybe the guessing of
modern-day individuals, based on extremely gross modeling from a tiny
information base, would be enough to correctly predict some aspects!
**
-- Eliezer S. Yudkowsky http://intelligence.org/ Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT