**From:** Luke (*wlgriffiths@gmail.com*)

**Date:** Wed Oct 14 2009 - 09:37:42 MDT

**Next message:**Bradley Thomas: "RE: [sl4] Alan Turing's results are profound"**Previous message:**Toby Weston: "Re: [sl4] Complete drivel on this list: was: I am a Singularitian who does not believe in the Singularity."**In reply to:**Pavitra: "Re: [sl4] what's with all the math?"**Next in thread:**Pavitra: "Re: [sl4] what's with all the math?"**Reply:**Pavitra: "Re: [sl4] what's with all the math?"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ] [ attachment ]

@Pavitra:

Many points, presented in no particular order. Bullet points are great for

that.

- I've read some of GEB (Hoftstader) and some of The Emperor's New Mind

(Penrose), so I'm familiar with the "formal" definition of "formal

mathematical" - i.e. not just 2 + 2 but also !Q => !P, etc.

- I may not have specified levels correctly on that task in version 0.

When I said "When will we know we have succeeded?" I should have said "When

will we know we have succeeded in creating a friendly AI?", and not "When

will we know we have succeeded in creating design requirements?"

- I kinda like Yudkowski's definition of intelligence as "that which hits

a target in less than chance time." This allows things like heat-seeking

missiles to be considered intelligent. Not mathematically rigorous, but

pretty close.

- I've debated with some MIT kids on the value of "understanding"

intelligence, and whether this is a necessary step in the creation of

intelligence. Being a zen buddhist, I'm incredibly skeptical that anything

can be fully understood. Yet I'm still able to get things done here and

there (though I'm always ignorant of the full effects of my actions). I

just want to invent a cheap, mass-producible neural network implemented in

hardware and sell it like the laptop-for-every-child machine, and have kids

developing a good intuition for what can and cannot be achieved with neural

networks, the same way they know what can and cannot be achieved with a

plasma grenade (see Halo franchise).

- I'm also a web developer, and I'm pretty confident that at least some

aspect of our future strong AI will be written in code. However, I predict

that the code will be simply managing tasks which are handled by black-box

systems like neural networks. Something like "if(@picture.contains("a

pizza")){ say "I see you're thinking about pizza - would you like me to

order you some?"}, where "contains a pizza" will be handled by a neural

network.

- Mathematical definition of friendliness - good luck with that. Perhaps

we could start with English definitions, then slowly start constraining our

language more and more until we can get a formal mathematical definition.

Does anyone know of a body of formal mathematical definitions of

sociological interactions?

- Being a civil engineer by training, web developer by trade, serial

entrepreneur, musician, masseuse, and meditator-at-large, and psychedelic

enthusiast, I see myself as bringing some project management expertise to

the table. I'd be curious to know what everyone else's background and

day-to-day line of work is. What do you all do? How can we forge ourselves

into an effective team, and not just a bunch of idly-chattering

idea-machines? Do we have any mathematicians in the group? Musicians?

Artists? Billionaires? Hippies? Politicians? Child psychologists?

I think there's a big tradeoff between rigor and applicability. Linear

systems (cannon firing at è from horizontal with velocity V - where will the

cannonball land?) lend themselves nicely to mathematical analysis - i.e. we

can answer the question without performing the experiment. Nonlinear

systems (6 * 10 ^ 23 concepts interconnecting and mutually

activating/inhibiting within a Jupiter Brain - will it try to kill us?) not

so much - i.e. we won't really know until we build the darn thing and turn

it on.

I'm really quite (1) optimistic about our ability to spawn intelligence

greater than our own, and (2) pessimistic that the resulting machines will

be so easily taken out with a simple EMP blast, let alone be guaranteed

friendly. But I'm willing to explore the space, because I think it's our

only hope.

But rest assured, I'm doing my best to live it up now because I don't have

much hope of living beyond 2040.

- Luke

On Tue, Oct 13, 2009 at 3:03 AM, Pavitra <celestialcognition@gmail.com>wrote:

*> Luke wrote:
*

*> > My question was this: You seemed to, for the most part, re-state a lot
*

*> of
*

*> > the steps in terms of a problem of formal mathematics. Why? Do you
*

*> > consider English to be insufficient/imprecise? Do we get to count
*

*> > ruby<http://www.ruby-lang.org/en/>as a formal mathematical system?
*

*> >
*

*> > Is there a formal mathematical definition of intelligence? What's so
*

*> great
*

*> > about formal mathematics?
*

*>
*

*> I don't know whether a formal definition of intelligence exists or is
*

*> possible. (I left the checkbox on step C unchecked, you'll notice.)
*

*>
*

*>
*

*> As for the value of math:
*

*>
*

*> This is a very difficult idea to explain, because it's very subtle. The
*

*> best hope I can offer you is that, once you get it, it feels utterly
*

*> simple and obvious in retrospect.
*

*>
*

*> What follows is a very poor attempt to allude at the general direction
*

*> of the answer you're looking for. If you're already most of the way
*

*> there, it may make some sense to you.
*

*>
*

*>
*

*> Yes, I consider English to be insufficiently precise. (Note, though,
*

*> that certain proper subsets of English can be used to communicate
*

*> mathematically-rigorous ideas.)
*

*>
*

*> Yes, I consider Ruby sufficiently formal. A Turing machine is specified
*

*> with mathematical rigor; a Turing machine (plus oracles for clock, HRNG,
*

*> etc.) can emulate a modern PC; therefore, any computer program is
*

*> mathematically rigorously defined.
*

*>
*

*> The existence of a formal definition of a question is a necessary
*

*> prerequisite to being able to answer the question with certainty.
*

*>
*

*>
*

*> Consider the following text from version 0 of the document:
*

*>
*

*> > Compile design requirements for "friendly AI". When will we know we
*

*> > have succeeded?
*

*>
*

*> In order for this step to be properly completed, the design requirements
*

*> must be so clear and precise that there is no possible dispute over
*

*> their interpretation.
*

*>
*

*> (There may of course be dispute over whether they correctly ask the
*

*> "friendly AI" question, but that problem is probably intractable even in
*

*> principle.)
*

*>
*

*> This degree of total precision is, essentially, the definition of
*

*> mathematical rigor.
*

*>
*

*>
*

*> Mathematics is more than just the ability to manipulate numbers. It's
*

*> _logic_, the discipline of clear, precise, unambiguous thought. Logic
*

*> means seeing the world with sharp edges, and distinguishing with
*

*> absolute certainty between truth and falsehood.
*

*>
*

*>
*

**Next message:**Bradley Thomas: "RE: [sl4] Alan Turing's results are profound"**Previous message:**Toby Weston: "Re: [sl4] Complete drivel on this list: was: I am a Singularitian who does not believe in the Singularity."**In reply to:**Pavitra: "Re: [sl4] what's with all the math?"**Next in thread:**Pavitra: "Re: [sl4] what's with all the math?"**Reply:**Pavitra: "Re: [sl4] what's with all the math?"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ] [ attachment ]

*
This archive was generated by hypermail 2.1.5
: Wed Jul 17 2013 - 04:01:05 MDT
*