From: David Picon Alvarez (eleuteri@myrealbox.com)
Date: Wed Dec 14 2005 - 13:42:24 MST
From: "Jef Allbright" <jef@jefallbright.net>
> David makes good points here, but interestingly, as we subjective
> agents move through an objectively described world, we tend to ratchet
> forward in the direction we see as (subjectively) good. Since we are
> not alone, but share values in common with other agents (this can be
> extended to non-human agents of varying capabilities) there is a
> tendency toward progressively increasing the measure of subjective
> good.
That's only a consequence of relatively symmetric game theory situations.
Subjective good is rising for us humans, because we're playing a symmetric
game and a certain level of cooperation is desireable for ourselves.
Subjective good isn't, or needs not, be rising for cows, which are playing a
completely assymetric game with us, we eat them, whether they like it or
not.
> Appreciating and understanding the principles that describe this
> positive-sum growth would lead us to create frameworks to facilitate
> the process of (1) increasing awareness of shared values, and (2)
> increasing awareness of instrumental methods for achieving our goals.
That would essentially come to game theory. A super AI probably would be
also assymetrically placed with respect to us. Our consent or cooperation is
probably not necessary or even helpful to an SAI.
> This paradigm would supersede earlier concepts of morality, politics
> and government.
The idea that agents create the good by approximatively taking good actions
which benefit them is the idea underlying extant systems in philosophy and
politics: contractualism, applied to whichever field, gives us
game-theory-based moralities (evolution of altruism kind of cases), the
Machiavelian idea of the Prince which acts in accordance to certain
limitations out of entirely selfish motives, or modern ideas from Rawls,
etc. To an extent, the legal theories on natural rights are also based on
this premise.
--David.
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:54 MDT