Re: [SL4] Employment vs. Singularity
From: Brian Atkins (brian@posthuman.com)
Date: Sun Aug 20 2000 - 17:15:44 MDT
Samantha Atkins wrote:
>
> Brian Atkins wrote:
> >
> > > That is irrelevant to this discussion. In very short order, if most of
> > > our ideas/notions are correct, most places will make Silicon Valley look
> > > like a sleepy backwater. Silicon Valley like effects are right now
> > > beginning to hit many areas of the country. If you check the average
> > > buying power of most of the middle class right through all but the top
> > > of upper middle class, you will see that very little unemployment slack
> > > will/can be taken up by hiring servants. And of course there is the
> > > small truth that most people are not cut out to be servants.
> >
> > Not cut out for it? I don't buy that; I myself worked in some service-
> > type jobs when I was a teen. Anyone can work those jobs if they can't
> > find anything else to do.
>
> I disagree on two counts. One, we should be able to employ our growing
> wealth and abilities to enable people to do what most suits them and
> most optimizes their personalities and talents, not simply whatever
> someone will pay for. People were meant to be a bit more than salary
"meant"? Are you implying some kind of creator?
> whores imho. I don't think it serves us well to not optimize for each
> person contributing what they do best regardless of whether we have a
> job open for it that pays in the conventional sense or not. What you
> would/could put up with as a teen and what a highly trained but
> currently unemployable physicist can/should put up with are very
> different.
Sounds like what the communists wanted/promised. It sure isn't ultra-
capitalism as I know it. How exactly would the jobs be provided for the
"worthy" if no one wants to hire them? Government subsidies backed by
a massive tax and spend program? I'm sorry, but the world (at least pre-
Singularity) is not perfect. If someone can't learn what they need to
learn to get a "good" job in the new economy, then they are going to
have to take whatever work they can get.
>
> Second, I disagree that everyone can do well whatever they decide to do.
Where did I say that? All I said was that they could do it- not that they
would love it, or be excellent at it.
>
> >And as for costs for the middle class, I don't
> > buy that either. If you want to quote me something showing they have less
> > spendable income I would like to see it. As far as most people in this
> > country are concerned, this is the best they've ever had it. I myself
> > live in Atlanta which also is experiencing a real estate and population
> > boom, but we have the option here of growing out into the suburbs. I
> > would argue that such places as San Francisco and NYC are artificially
> > limited by geography, and therefore have no way for the middle class to
> > survive there long term.
> >
>
> I don't have quotes at hand for you of the actual lower buying power in
> some critical areas like housing that the middleclass has today. Those
> studies exist. I simply don't have the time to look them up right now.
Here is a brief blurb about burgeoning middle class buying power in asia:
http://www.edge.org/3rd_culture/story/76.html
Here is a blurb supporting the idea that the middle class is getting
shafted in the new economy- productivity gains are not leading to massive
increases in pay:
http://www.epinet.org/webfeatures/viewpoints/laborday%2099.html
I will buy that argument, but note it does show that they did experience
SOME (albeit small) growth in wages over the last 10 years. Also note that
inflation in general has been abnormally low during that period. At any
rate I don't know why we are discussing buying power since it is off-
topic. We should be talking about unemployment, not the excellent efficiency
of our current economy.
Finally keep in mind as I said, quite a lot of people in the last 10 years
escaped the middle class to go higher. Just because the middle class is
getting shafted doesn't mean that a lot of them are getting rich.
>
> > >
> > > > Anyway, we'll see about all this as time
> > > > goes by. The middle class may not get back to the level of service they
> > > > enjoyed in the 1800s, but they already make enough money to use many of
> > > > the new services that are becoming available. They may not have full
> > > > time servants, but they will still provide more and more jobs in the
> > > > service areas.
> > > >
> > >
> > > Not enough by far. This is not the 1800s and trying to use that as a
> >
> > I didn't say it was the 1800s
> >
>
> No, you simply used it to mark your point. It isn't very good for that
> purpose.
>
> > > fit or even suggestive analogy is pointless. At some point and perhaps
> > > already we must deal with the problem of human beings who are
> > > unemployable and unneeded in the hyper-technological economy. That is,
> > > we must if we care about human beings at all and are actually able to do
> > > anything about the problem. We can, of course, assume this happens only
> > > to Them and will not happen to Us as we are so bright and well-fortuned
> > > and so on. But that is a very self-centered attitude that fully
> > > justifies and can expect quite a bit of backlash. It also paints a
> >
> > Well I think you should break your response into two different subjects
> > when you respond. One for discussing whether or not we actually will
> > have an unemployment problem, and the other for what to do about such a
> > theoretical event. What would be your solution to the hypothetical problem?
> >
>
> Figure out as quickly as possible how to ensure all have a workable
> standard of living including the prime necessities and room to grow from
> there regardless of whether we have a standard paying job for all of
> those people. As we get closer to nanotech and as technology advances
May I suggest you move to Sweden? I hear they have a 50%+ tax rate for
the productive individuals, and the highest percentge of leeches (oops,
I mean "people without a standard paying job") in the world.
> we become better able to produce all the necessities and many of the
> wants for everyone with increasingly less human labor involved. I don't
> think I need to quote studies for that point to be pretty obvious. If
Well it isn't obvious to me why penalizing (through higher taxes) the people
who actually go through the trouble to learn how to work in the new economy
in order to support the losers is such a good idea.
> that is so then it is logical that we find ways to distribute these
> goods and services that are less dependent on the recipients being
> employed producing them. As we go forward we should be able to acheive
> greater human freedom rather than following the same "wage-slave" model
> that developed in a very different past than the present, much less the
> near future.
That sounds nice, but can specifically describe your future scenario
showing me how all this is going to work?
P.S. you do realize that your average person who can't get a job tends to
get depressed and unhappy? They want something to do, and if watching TV
is all they have, they tend to head off sometimes into self-destructive
behaviors. If we rush to put 50% of our workforce out of work just "because
we can" and we "want to increase happiness", we may the exact opposite
effect for many people.
>
> > > pretty dismal picture for what we can expect from the hyper-intelligent
> > > AIs if they come on the scene if we assume that such an attitude is the
> > > most logical and intelligent way of dealing with such problems. By the
> > > same logic they will simply ignore us and use most/all of their
> > > resources only for themselves. The result will be our extinction.
> >
> > Well you just made this up, and I'm not sure why. If we have hyper-
> > intelligent AIs then this whole argument is moot- that would be post-
> > Singularity and we really can't say for sure what it would be like.
> > Let's stay on the two topics.
> >
>
> No. I did not make it up. It has been said on this list that those who
> aren't making it are to blame for their condition and that anyone who
> wants to make it can. It has been suggested that people can slot
> themselves into whatever jobs are paying money and that thus there is no
> problem. If the AIs take this attitude and have no real use for us at
> all or a use that is distasteful and wasteful of human abilities and
> lives then we will not have a very happy ending to our long rise from
> the cave. It is not just one or two topics. These issues are quite
> throughly inter-related and entangled.
Well now you are heading off into a completely unrelated third subject:
will AIs eat us all. I am having trouble keeping this conversation going
if we are going to keep branching. I again suggest when you reply that
you create multiple new subject lines each addressing your points:
AIs may eat us all
As singularity approaches it will create massive unemployment
Suggestions on how to handle theoretical singularity-caused unemployment
Not only will that make this all easier to read, you might get some other
people jumping in on specific topics.
>
> The argument is not moot because, hopefully, we can manage to get to the
> singularity with some confidence and buy-in that we are actually headed
> for an improved reality rather than simply our demise. Not many people
> are going to buy in on a hyper technological pig-in-the-poke that might
> produce heaven on earth and then again it might wipe earth and all of us
> out of existence before it really figures out what it is and what it
> wants. If we can't figure out what our right relationship is to other
> human beings then it is pretty unlikely we can expect AIs to figure it
> out or especially to come to a relationship with us that we can expect
> to be at all benevolent.
Well the fact is, unless the government comes and arrests us all I don't
see the people being able to stop our research. We (SIAI here) have spent
a great deal of time thinking over the alternatives such as:
let's wait for nanotech (planet wipe-out more likely due to human error or
terrorism)
let's wait for uploading (first human uploads probably take over the planet)
let's take the Bill Joy approach and stop (very very bad, 150k people dying
every day we don't reach the Singularity! slow down is bad news)
and come to the conclusion that creating a Singularity ASAP is the best
possible choice. It's true that we can't fully cover all the risks in
developing a real, intelligence-enhancing AI mainly because we aren't smart
enough to know what a super-intelligence will do! On the other hand I think
you've watched The Matrix too many times- a SI AI will likely have little
to no use for us humans, just as we have no real use for ants. It may go
jetting off into the universe and not do didly squat for us. We HOPE it
will give something back in return for creating it.
However using humans -> ants as an analogy to SI AI -> humans is a bad
example I have to stop using. A SI AI will be aware that we are intelligent,
and would not even consider treating us like we treat ants. It would not
consider eating us or our planet unless it was completely insane- and that
is a factor that is completely under our control as we design, build, and
hopefully test the seed AI.
>
> > > Your contention is simply an assertion and one that is increasingly
> > > unlikely as the technological basis of all economic activity becomes
> > > increasingly advanced and automated. In the not unimaginable near
> >
> > Really? So far your contention is also an assertion, and based on our
> > experience of the last 100 years (which I also consider to be part of
> > the run-up to Singularity) you would be wrong. So by Occam's Razor can
> > we also guess that I may be right, and that our economic progress and
> > low unemployment rate will continue? You on the other hand sound like
> > some of the people around 1900 who were so worried about the age of
> > machinery. (to be blunt)
> >
>
> Yes, but I've tried (as you have) to back up the contention with
> reasonable argument that you have not addressed adequately imho. We
I have pointed out above that over the last 100+ years things have only
gotten better employment-wise as tech has developed. You have not pointed
out a single fact showing an increase in unemployment now, or showing why
we will have an unemployment problem in the future.
> are not facing something that looks like the last 100 years that much
> unless you condense that amount of change at least 10-fold (this year).
Ok, but on the other hand if you agree with me that the last 10 years
during which our economy has done so well was also not like the previous
90 years, then my argument holds. You have not given any proof that the
workers in our country will not be able to deal with increasing rates
of change. In fact many of them seem to thrive on it!
Not only that, but if you do make the claim that there is a certain level
of rate of change that workers can't deal with, then I can use that to
say that the runup to Singularity will level off at that rate until we
develop enhanced intelligence. i.e. the economy will self-regulate itself
to sustain only the maximum rate of change that the workers can deal with.
So what are you worrying about?
> Even to take that example, how many different economic forces and
> dislocations of workers and economies have occurred in that time? Do
> you notice that this post-industrial problem was first identified in the
> 1950s (by Galbraith I believe)? Did you not notice that the US and most
> western countries introduced massive welfare programs to attempt to take
> up some of the slack of people not just naturally being reasonably ok
> due to working in the economy?
Sure I noticed- I also noticed an article I read in the paper a couple of
weeks back that pointed out we are now at record lows in many states
regarding the number of people on said welfare. I just don't see the
doomsday you are pointing at for all these workers.
>
> Occam's razor is not applicable in your above argument. Your theories
> are not simpler and even if they are that is not an argument that they
> are more correct. Insulting me by implying I am a Luddite is
> especially foolish. I am quite the opposite. But I refuse to assume
> just because we are exhibiting higher technology that that will
> automatically result in a better world. Do not confuse an attempt to
> envision how to get from here to there with the least pain and the best
> result with a claim that we shouldn't go there at all.
Ok, but I don't see your version of the interim meaning of life. I don't
see your masterplan. Is it on a webpage somewhere? What do you think
exactly is the best plan to use over the next 10 to 20 years pre-Singularity?
Lay it all out for us in detail.
>
> > > future un-augmented human beings will not even be able to compete in the
> > > intellectual white-collar areas of the market. With what are the
> > > unemployed and unemployable to obtain such augmentation? In some areas
> >
> > Umm, well you do you live in the Valley, you should know that the companies
> > there will do pretty much anything to get employees. In fact today we
> > already have companies like Ford and Delta giving away computers to their
> > employees. I think it is obvious that future companies will give their
> > employees whatever tech and augmentation is required. So bzzt on your
> > suggestion that tech will put info workers out of work.
> >
>
> That does not follow. Please keep rude noises to yourself.
That was not a response. Do you have a real response to my claim that the
companies will keep their employees upgraded with whatever they need to
work?
>
> > > of intellectual work you don't even need human level AI to replace human
> > > workers. You simply need computers designed/programmed for focusing
> > > deeply on the delimited problem at hand to the extent they are superior
> > > in cost/benefit to the humans who formerly performed that function.
> >
> > Can you be more specific as to exactly what jobs here you are referring to?
> >
>
> Specifics are not terribly relevant when attempting to point out the
> obvious.
Another no-response. How can we have a debate without specifics? Just one
or two is all I ask.
>
> > >
> > > Car companies in the states at least are blocked from full automation by
> > > unions. But this situation is quite temporary. Much of the
> > > manufacturing jobs in the US already have either been automated or are
> > > now filled where the labor pool is cheaper.
> >
> > Unlikely, since car companies and by extension their parts suppliers have
> > a vested interest in producing "made in america" cars. Even foreign car
> > companies like Honda, Mercedes, and BMW produce cars here in order to
> > get that distinction.
> >
>
> I think you missed a lot of recent history. "Made in America" is no
> longer considered a good thing when it comes to cars. It hasn't been
> for some time.
Well I was pointing out recent history: foreign car makers very recently
(last 3 years for BMW/Mercedes) opening new factories here in the US. And
I have a hard time coming up with news articles pointing out large factory
closures by the Big 3 automakers in the US. You whole argument is based on
things you forsee happening in the future, but there doesn't seem to be
much you can point at to back it all up.
>
> > >
> > > Who cares? Anyone that cares to really mark the trends and cares about
> > > the human beings being left by the wayside and cares what that may
> > > portent. How the numbers are calculated, afaik, has been changed many
> > > times since the sixties. Pointing to this one indicator still does
> > > nothing to address what looks to me like an inescapable problem as we go
> > > forward.
> >
> > Sure it does, it shows that at least since the last time the indicator
> > was changed, our unemployment is at a record low. Any yet by your reasoning
> > we should already be deeply into unemployment issues since our infotech
> > revolution has been going on for over 30 years. When exactly do you see the
> > big jobs collapse occuring?
> >
>
> Are you ignoring how many people are on the dole here and in the other
> Western nations?
I just pointed out above that welfare recipients in many states are now
near all time lows. I can dig the actual articles up on this if you want?
As for other western nations (I guess you mean Europe) that is not
comparable to the US. The mess they have is their own political/social
making, and not due to technology. In fact, their lack of use of the
Internet and other technologies is holding them back!
>
>
> > >
> > > > > >Did
> > > > > > you read the sci-fi book The Diamond Age? Our society might move towards
> > > > > > some kind of neo-victorian mode before the Singularity wraps things up.
> > > > >
> > > > > And what will this wrapping up be like in your view? Do you think human
> > > > > displacement will only occur during full-bore singularity or will some
> > > > > of it take place along the way to getting there?
> > > > >
> > > >
> > > > Oh, when I say wrapping up I mean the Singularity itself- i.e. technology
> > > > suddenly (probably due to a real AI finally waking up) begins changing at
> > > > a near infinite speed, and within a very short time life as we know it is
> > > > over with (unless you want to stay in a human body on this planet, in that
> > > > case I'm sure the AI will let you do that). Read Diaspora by Greg Egan if
> > > > you haven't- I'm taking about an Introdus.
> > > >
> > >
> > > I have read most of those books. But my question is, how will we treat
> >
> > Did you read Diaspora?
Another no response. I take that as a no.
> >
> > > one another, what is the proper way to treat one another morally, as we
> > > become more and more advanced? Is there a way of making sure as many as
> > > possible win? Is that way global capitalism, or current mixtures, or
> >
> > Yes, by accelerating the Singularity (but hey, I'm biased :-)
> >
> > > something yet to be found and implemented? Is it actually critical to
> > > our success in reaching the singularity and to the maximum efficiency of
> >
> > I think this subject is important because yes it is important in terms
> > of reaching the Singularity that we maintain at a minimum our rate of
> > tech development and society stability.
> >
>
> Where will this stability come from if I am even moderately right and
> more and more people feel disenfranchised from the society and more and
> more uneasy with where it is going? You need buy-in as you go as much
> as possible. Do you think that a society where half the population
> believe in bogey-men (angels, demons, young earth and so on) and where
> the average college graduate reads only one book after graduation in
> their entire life is stable? Especially at a time of accelerating
> technological change? Dunno. It looks to me like we aren't doing such
> a bang-up job in a lot of areas that we can just congratulate ourselves
> and call for more of the same.
Well yes, if your employment-collapse theory happens that would BAD. We
are discussing that above, so I won't go into it here. What I will point
out, is that assuming our employment level remains ok, I don't see why
we will have a huge backlash pre-Singularity. Remember I talking by 2020
here. In that time span, there are two main scenarios I guess to consider:
no uber-nanotech, or real nanotech. Assuming no major nanotech-in-our-
daily-lives-ala-Diamond-Age then I don't think much will change for the
average person. They will have ever-faster computers, improvements in
their lifestyle/span due to biotech advances, and plenty of new toys to
keep them entertained. But there will not be any major life-changing
things going on. Things like the Singularity will seem to be far-out
schemes that likely won't be taken that seriously until near the very
end of the pre-Singularity era. In a world with nanotech it is a very
different story- that is really hard to guess what it might look like.
It might be tightly regulated so that it kind of slowly filters into
the lives of people. Or depending on how easy it is to do, it might
become very widespread like computers are today, allowing any joe
schmo to create stuff in vis basement. Scary thought.
Assuming a no nanotech-everywhere scenario, what do think will people
find so shocking about the next 10-20 years?
>
> > > any post-singularity society of whatever types of intelligences that all
> > > individualities are cared for enough to maximize their abilities and
> > > outputs (sort of a la STNG)? Or do we believe that endless competition
> > > (which seems to me to be more of a scarcity game) is the only or most
> > > intelligent play that we can/should expect going toward (and hopefully
> > > through) the Singularity?
> >
> > No, we (I mean SIAI here) believe that there should be some amount of
> > resources guaranteed to everyone post-Singularity. Things should start
> > out equal. If you look over at the archives of the FoRK list (www.xent.com)
> > there was a debate between Eliezer and some other people on just this
> > subject a couple of days ago.
> >
>
> OK. Something we at least in part agree on. :-) Although I think it
> goes beyond simply starting out with equal resources if you wish to get
> the maximum good from each being that they are capable of producing.
>
> > >
> > >
> > > > Lastly let me lay out my timeline, I place the Singularity no later than
> > > > 2020. From now till then I don't see massive unemployment, due to the fact
> > > > that service jobs and technology jobs will continue to increase, and the
> > > > only automation will continue to be in areas where there are no strong
> > > > unions, and jobs are simple enough to be done by dumb machines.
> > >
> > > You can't get there from here. There must be ever increasing machine
> > > intelligence and automation all along the path if we are to reach 2020.
> >
> > I disagree, the only thing that we (SIAI again) need to get our Singularity
> > done is a certain base level of computational power (which we will probably
> > have no later than 2010, and which does not require any form of AI to
> > develop), and the programmers and other scientists to develop the code
> > to run on it.
> >
>
> We wil have several years of increasing technology and its
> ramifications on society to go through to get there. I don't think your
> agreement or lack of same changes that. This software needed will not
> spring full-grown out of nowhere any more than the computational power
> will.
We are working on the necessary software. That will come from us. The
hardware is the most important unknown for us, but it simply looks more
promising every day. We might have the needed horsepower as soon as the
late 00's.
>
>
> > > One or two advances in computational density without advances in
> > > software and other areas (for instance) will not get us there. It is a
> > > continuous curve (more or less) up to that point. The above doesn't
> > > seem to acknowledge points that I believe are inevitable along the way.
> > > We will see many service jobs (I think Moravec mentioned fast food
> > > workers once) disappear along the way. We will see levels of home
> > > automation and new products that make personal servants much less
> > > likely. We will see the unions cracked open (already happening) as they
> > > become increasingly a drag on hyper-production. The machines will and
> > > must become ever increasingly less dumb if we are ever to arrive at
> > > singularity.
> >
> > I think you're right that the curve will continue heading up (even though
> > we don't need all of it), but on the other hand I still don't buy your
> > argument of an unemployment disaster. Who is going to design, build, sell,
> > install, and maintain all these wonderful new toys? The more toys we
> > invent, the more jobs there will be created. Even if some services do
> > eventually lose jobs, I don't see the collapse you are predicting. I'll
> > have to go see if I can find any kind of academic works to back up my
> > assertion, or perhaps someone on the list has something to say.
> >
>
> There will of course be engineers, scientists, marketing folk and so on
> involved in producing and exploiting the coming advances. That
> certainly does not say that everyone will be employed in the needed
> capacities. I did not predict a collapse actually. I just predict a
> growing problem that it is important for us to address well.
Ahh ok, my bad. But if it is just a slowly growing issue over the next
couple of decades I am not too worried about it. Our government will
simply put them on the dole, end of story. No big worries there. We are
only worried about massive things that might slow down the development
of the needed computing hardware.
>
>
> > >
> > > >As the
> > > > citizens of the world get gradually richer, demand for these services and
> > > > luxury products will continue to rise, creating jobs in service and artisan
> > > > areas. Etc. etc.
> > >
> > > If we continue with the current model then fewer and fewer people will
> > > get rich enough relatively to hire the others and fewer of them will
> >
> > Whaaa? We have more millionaires here in the US than at any other time.
> > All I see is more and more people making more and more money.
> >
>
> We have more millionaires and the actual buying power of a million
> dollars in considerably less than it was at mid century or even a few
> decades ago. You can have wheelbarrows full of money and still have a
> very lopsided distribution of wealth where only a few can hire that many
> others (as you suggested would occur).
Well in fact last I heard, the service side of our economy is still I
think the most rapidly growing in terms of number of jobs created.
All the middle class won't have their own servants, but they will continue
contributing to this increase in service jobs IMO.
>
> In some types of real wealth we are much better off. There are many
> types of wealth now common (information technology based mostly) that
> could not be had at any price before. But the effect of these great
> liberators has not been as much liberation of human throught and
> creativity as it should have been.
Well we will just have to see how the whole intellectual property/Napster
thing works out won't we? I think you are a little early to say that
tech killed creativity.
>
> > > need/want human beings to perform tasks that can be more efficiently
> > > handled by automation. The residue of things that only humans can do
> > > will shrink over time. I see no way current competition based (and
> >
> > It will very slowly shrink, I would predict along the same lines as the
> > Singularity itself proceeds.. there will be a slow growth in the types
> > of things not done by humans, until just before the Singularity at which
> > point many things will no longer need to be done by humans (although they
> > still might want to do them!). But just because some tasks continue to
> > be automated does not imply an unemployment disaster. Think about how
> > many tasks have already been automated since 1900, yet we have no problem
> > finding jobs for practically everyone?
> >
>
> Count the millions on the dole and also count the umemployment
> statistics in other industrialized countries before you say we have "no
> problem finding jobs for practically everyone". When you add it up I
> think you will see we have a large problem finding jobs for something
> closer to 18% of the adult population than to the official 4% unemployed
> in the US.
Nevertheless as I reiterate: the unemployment number is at an all-time
low, the other countries problems are their own creations, and in many
areas of the world such as Asia there is a huge creation of new higher
paying jobs.
>
>
> > > actually scarcity based) economic models can continue to run going
> > > forward without displacing and disowning more and more people. I would
> > > very much like to see it differently.
> >
> > Well we see an example of this here with MP3s/Napster/etc. It remains to
> > be completely resolved, but most people seem to think that the artists
> > will still be able to make a living even after the scarcity of the data
> > distribution is gone. The issue comes down to human creativity I guess-
> > /that/ will still be scarce until such time as we have full strong AI,
> > and at that point we will have the Singularity.
> >
>
> Not exactly what I was getting at but you bring up an interesting
> point. That point being that information is increasingly the basis of
> our wealth making abilities. And information wants/needs to flow
> relatively unimpeded if it is going to reach its maximum value as
> information. We see this increasingly in the Open Source software
> world. Software, many argue, is best developed and maintained when it
> is open rather than when it is owned by particular entities and kept
> closed in its particulars (information). Nanotech will be based on some
> core technology (assemblers) but will largely be able to produce
> miraculous results based on information concerning how to build various
> desired things and more information flowing between nanobots, foglets
> and so on. Old model business entities based on controlling/owning
> resources get in the way when the resource is information. But this
> takes us into an entirely new layer of the onion.
Hehe, yes a fourth potential subject line! Who can say how this will
develop exactly.
>
> - samantha
>
>
--
Brian Atkins
Director, Singularity Institute for Artificial Intelligence
http://www.intelligence.org/home.html (temp site)
This archive was generated by hypermail 2.1.5
: Wed Jul 17 2013 - 04:00:35 MDT