From: Tennessee Leeuwenburg (tennessee@tennessee.id.au)
Date: Fri Sep 09 2005 - 00:15:11 MDT
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
> Now the curious thing about AGIs are that they are the ultimate in
> state compression functions. When Eliezer talks about building a
> 'really powerful optimisation process', he is talking about
> building a system that will (reliably) squeeze all the probability
> mass in the PDF over possible future histories into a small subset
> that the goal system defines as desireable. This is the very
> definition of 'goal seeking process'; a process which predictably
> tends to steer the state of the world into desireable states,
> though the intermediate states and means it uses to do that may not
> be so predictable.
I think you'll find that is in line with my description of Chaos
theory. The system is deterministic, but the details are difficult to
predict with anything less complex than the system itself.
I have heard this argument run with regards to the universe itself.
Would it be possible to build a computer which could simulate the
physics of the entire world? Ignoring the contradiction of something
predicting its own predictions, would the mechanics of the universe at
large be possible to track with anything less complex than the
universe itself?
This, in some sense, is also the challenge faced by futurists and AI.
It suggests some pretty hard practical limits to AI's predictive power
in relation to actual external events, even if the deterministic rules
governing them are known.
I don't think it's possible to treat the engineering of a complex
systems as just a matter of building from foundational tools with
predictable behaviour. When a less complex system (us) tries to
predict the precise behaviour of a more complex system (the AI), there
is always error, approximation, or a loss of detail.
Cheers,
- -T
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (MingW32)
iD8DBQFDIShvFp/Peux6TnIRAk89AJ9+a7XQc0ZY+UTIOyHWjAxW3Fe6EACfTg4a
GcU6V2zDAKzs8y9853GABSU=
=0GUm
-----END PGP SIGNATURE-----
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:52 MDT