From: Samantha Atkins (samantha@objectent.com)
Date: Sun Jun 13 2004 - 21:27:16 MDT
On Jun 13, 2004, at 3:32 PM, Ben Goertzel wrote:
>
> Hi,
>
>> So now sufficient POC to guarantee the SI will not take off and do
>> terrible things depends on humans reading and verifying a
>> specification
>> of the intent of the programming? This doesn't look very "safe" to
>> me.
>
> Hey you -- you're completely misrepresenting what I said!
>
I don't think so. You just said that some program description in a
language like Z would be hand verified by humans before being fed to
the program verifier. This seems to introduce more than a little risk.
> If you read my original post, I was describing
>
> * a layered architecture for minimizing the already small risk involved
> in experimenting with infrahuman AGI's
>
> * the possible use of formal verification tech to verify the
> correctness
> of this layered architecture
>
Well, so far you can verify that the program corresponds to the
description of intent written in Z and verified by as a reasonable
description by the same fallible humans who wrote the program. I
don't see how you can get a meaningful "verification of your entire
layered architecture" from such a process.
> I was not positing a long-term AI Friendliness architecture, rather an
> architecture to improve the safety of running experiments with
> infrahuman AI, which will help us gain knowledge about AI and
> Friendliness that will help us to build the theoretical tools needed to
> create a good Friendliness architecture.
I did in fact understand that. I do not understand your assurance that
the process described is sufficient to even this limited task.
- samantha
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT