From: Nick Tarleton (email@example.com)
Date: Fri Apr 25 2008 - 15:50:38 MDT
On Fri, Apr 25, 2008 at 10:31 AM, Tim Freeman <firstname.lastname@example.org> wrote:
> On Thu, Apr 17, 2008 at 11:59 PM, Nick Tarleton <email@example.com> wrote:
> > Fixing who the AI cares about is over-specification. That's what the
> > AI (in the CFAI model) or extrapolated volition (in the newer model)
> > is supposed to figure out.
> > http://www.sl4.org/wiki/CoherentExtrapolatedVolition
> CEV fixes who the AI cares about. Quoting directly from the cited article:
> >As of May 2004, my take on Friendliness is that the initial dynamic
> >should implement the coherent extrapolated volition of humankind.
> The AI cares about the extrapolated volition of "humankind", not the
> extraploated volition of mammals or some other group.
The extrapolated volition of humankind could choose to extend the
group. The selection of humankind is part of the *initial dynamic*,
it's right there. If you fix humanity (or present humanity, or
whatever) as part of the goal system/utility function, it will never
change, because a rational agent resists changes to its
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:02 MDT