--- Dani Eder wrote: >Given the propensity for people to let the AI out >of it's box, for safety's sake the seed AI hardware >design will need a requirement of "no way for a >human to let the AI out". For example, the computer >hardware contained within a sealed box which when >opened will release acid to destroy the hardware. >Output devices (like monitor screens) behind glass >windows. The room they are in full of nerve gas, >so any attempt to hack the output devices is very >difficult, etc. We discussed this at length >previously, but it seems like we need to include >the safety provisions in a seed AI installation. Well, the problem as I see it, is not that we always let the AI out of the box, but that we can't accurately judge whether it is friendly or not. Becuase the precautions have to include someway to let the AI out if it's nice, otherwise, why are we making the AI? just so we can wave at it through soundproof, auto-opaquing glass? Also, the big problem is not just people letting it out, but the AI acting on the world in an unFriendly way.. so it would have to be a prison for the AI as well, and the difficulty of imprisoning a transhuman intelligence, even one of known origin and substrate, is difficult to assess at best, and impossible at worst. The challenge is not to design a perfect prison or vault, but to create an AI that we can trust to some reasonable degree, whether through character or design. Now character is shifty, and while i'd love to trust ver on that, I'm afraid I probably couldn't take the chance. So design it is. And at this point, all I've heard are Asimov Law-type injunctions, Friendly Goalsystems, and Goertzel's Novamente's FriendlyNode thingey. Of those, Friendliness(a la Yudkowsky) and perhaps Goertzel's sytem are the only types I think would survive, although i'd have to see more of the documentation on both in their specific instances of course. Most other schemes depend on determining the character of the AI, which is so insanely dangerous and ill omened, I hardly have words for it. (well, of course I have words for it, I just told you some, don't let my propensity for hyperbole scare you off, it's just really really a bad idea) An AI will not be a person like a human is. It's personality and goals are likely to follow rules unintuitive and alien to us. Our social reflexes are entirely inappropriate. And very misleading, as the AI boxing experiment illustrates. Absolutely none of the participants let the AI out for the right reasons, and I believe that to be because the right reasons can't be determined by interaction at that stage and in that medium. Justin Corwin outlawpoet@hell.com "No, I don't think so. A person can be sincere and still be stupid"