From: Ben Goertzel (ben@goertzel.org)
Date: Wed May 10 2006 - 20:45:57 MDT
> This doesn't necessarily impose important real-world limitations
> because, as I remarked earlier with the bridge analogy, an engineer
> operates within the subspace of systems that knowably work.
Yes, I understand this point... and I remember we've been over this before ;-)
I don't think the analogy of bridges is a very good one, because the
important aspects of bridges can be understood to within a high degree
of approximation by models with much lower algorithmic information
than the human brain ... whereas the important aspects of superhuman
AI's may well not be understandable to within a high degree of
approximation by models with lower algorithmic information than the
human brain...
My intuition is that important real-world limitations *will* exist. I
suspect that "being provably Friendly by a much stupider system" is
going to prove a verrry profound constraint on the amount of
intelligence a system can achieve given a fixed amount of resources.
But I don't know how to prove this...
-- Ben
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT