From: Jef Allbright (jef@jefallbright.net)
Date: Wed Dec 14 2005 - 14:41:33 MST
On 12/14/05, David Picon Alvarez <eleuteri@myrealbox.com> wrote:
> From: "Jef Allbright" <jef@jefallbright.net>
> > David makes good points here, but interestingly, as we subjective
> > agents move through an objectively described world, we tend to ratchet
> > forward in the direction we see as (subjectively) good. Since we are
> > not alone, but share values in common with other agents (this can be
> > extended to non-human agents of varying capabilities) there is a
> > tendency toward progressively increasing the measure of subjective
> > good.
>
> That's only a consequence of relatively symmetric game theory situations.
> Subjective good is rising for us humans, because we're playing a symmetric
> game and a certain level of cooperation is desireable for ourselves.
> Subjective good isn't, or needs not, be rising for cows, which are playing a
> completely assymetric game with us, we eat them, whether they like it or
> not.
>
> > Appreciating and understanding the principles that describe this
> > positive-sum growth would lead us to create frameworks to facilitate
> > the process of (1) increasing awareness of shared values, and (2)
> > increasing awareness of instrumental methods for achieving our goals.
>
> That would essentially come to game theory. A super AI probably would be
> also assymetrically placed with respect to us. Our consent or cooperation is
> probably not necessary or even helpful to an SAI.
>
Thanks David for highlighting the necessity of near-symmetry between
agents. I was going to mention this later in the discussion since it
is of critical importance to the question of whether we'll have time
to develop a broad-based collective intelligence augmented with AI
before we're made irrelevant by one or more narrowly focused SAIs.
Of course, this list has dealt with this particular question many
times already so I don't intend to bring it up for rehashing. My
intent was to clarify what we mean when we talk about "morality" in
general and the inconsistencies of conventional ethical thinking in
particular.
- Jef
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:54 MDT