From: Matt Mahoney (matmahoney@yahoo.com)
Date: Mon Sep 15 2008 - 08:35:46 MDT
--- On Mon, 9/15/08, Wei Dai <weidai@weidai.com> wrote:
> I wonder where the assumption came from, that groups as
> well as individuals
> should be guided strictly by expected utility maximization?
> Is it just that
> no one has proposed a viable and mathematically worked out
> alternative?
For example, a group of miners are trapped underground in an accident. It will take 48 hours to dig them out, but unfortunately there is only enough air for the group to survive for 24 hours. The individual preferences are that everyone lives, which results in everyone dying. However, the social preference is that half of the miners kill themselves immediately so that the other half will have enough air until they can be rescued.
-- Matt Mahoney, matmahoney@yahoo.com
> From: Wei Dai <weidai@weidai.com>
> Subject: Re: [sl4] Bayesian rationality vs. voluntary mergers
> To: sl4@sl4.org
> Date: Monday, September 15, 2008, 12:43 AM
> Tim Freeman wrote:
> > So after the coin flip, the utility function for the
> merged AI might
> > be "51 for a universe of 60% staples, or 49 for a
> universe of 60%
> > paperclips, or nothing if I don't get enough
> staples or enough
> > paperclips". Swap the numbers if the coin flip
> comes out the other
> > way.
>
> I haven't made much progress on finding another example
> that clearly
> supports my suggestion that the "period of
> irrationality" might not be
> short, but one thing that puzzles me here: why 51/49? Why
> not 52/48, or
> 50.1/49.9? It seems that the original AIs would prefer
> 50.1/49.9 to 51/49 to
> 52/48, but if they go all the way to 50/50, things fall
> apart. That seems
> quite strange.
>
> I tried doing a literature search to see if there has been
> any academic
> research that might apply here, and it turns out that the
> problems of
> merging differing priors and utility functions have been
> noticed by
> economists in the context of creating an aggregate
> prior/utility function
> for purposes of government policy and the like. There are
> several recent
> papers on merging priors, which all pointing to a 1955
> paper by Harsanyi [1]
> on the issue of merging utility functions. These works seem
> to show that
> merging priors is a worse problem than merging utility
> functions: whereas
> merging utility functions can only make some individuals
> worse off, merging
> priors can make everyone worse off. This is reflected in my
> examples as
> well.
>
> What Harsanyi showed was that if we assume the following
> postulates:
>
> A. Social preferences can be represented by expected
> utility maximization
> B. Individual preferences can be represented by expected
> utility
> maximization
> C. Society is indifference between two outcomes A and B
> whenever all
> individuals are indifferent between them
>
> then the social utility function must be a linear
> combination of individual
> utility functions. It apparently wasn't considered that
> postulate A should
> be false.
>
> In [2], the authors show the possibility of merging
> differing priors, but
> only by giving up the "Pareto condition" (similar
> to postulate C above) that
> when all individuals in society agree on preferences
> between two
> alternatives, so should society. Again I didn't see any
> consideration that
> perhaps social preferences should not take the form of
> expected utility
> maximization.
>
> I wonder where the assumption came from, that groups as
> well as individuals
> should be guided strictly by expected utility maximization?
> Is it just that
> no one has proposed a viable and mathematically worked out
> alternative? I
> didn't see anything in the literature similar to the
> "flip a coin" solution
> that several people have suggested here.
>
> [1] "“Cardinal Welfare, Individualistic Ethics, and
> Interpersonal
> Comparisons of Utility"
> http://www.unb.br/face/eco/jglresende/Harsanyi_CardinalWelfare_JPE55a.pdf
> [2] "Utilitarian Aggregation of Beliefs and
> Tastes"
> http://www.tau.ac.il/~samet/papers/utilitarian%20aggregation.pdf
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:03 MDT