**From:** Petter Wingren-Rasmussen (*petterwr@gmail.com*)

**Date:** Mon Feb 09 2009 - 11:26:19 MST

**Next message:**Vladimir Nesov: "[sl4] Weakening morality"**Previous message:**John K Clark: "Re: [sl4] Uploads coming first would be good, right?."**In reply to:**Johnicholas Hines: "[sl4] simplistic models of capability growth"**Next in thread:**Johnicholas Hines: "Re: [sl4] simplistic models of capability growth"**Reply:**Johnicholas Hines: "Re: [sl4] simplistic models of capability growth"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ] [ attachment ]

Defining things is a very good start :)

On Thu, Feb 5, 2009 at 2:27 AM, Johnicholas Hines

<johnicholas.hines@gmail.com> wrote:

*> 6. This last one takes a bit of motivation. Suppose the system in
*

*> question has two parts, one which is modifiable by the system and one
*

*> which is not. The total capability of the system is a combination of
*

*> the two parts. But the combination is not a simple addition or
*

*> multiplication of figures of merit. Rather, the total capability of
*

*> the system is measured in "speed", and each "elementary cycle" needs
*

*> to pass through both the modifiable part and the nonmodifiable part.
*

*> There is one endogenous variable "modifiable part's speed" (m).
*

*> There are two parameters, "modifiable part's improvement per
*

*> capability" (mpc), and "nonmodifiable part's speed" (n).
*

*> There are two equations:
*

*> The total capability of the system, the speed, is the inverse of the
*

*> sum of the inverses of the modifiable and nonmodifiable parts' speeds
*

*> (c = 1/((1/n)+(1/m))).
*

*> The time derivative of the modifiable part's speed is proportional to
*

*> the total capability (D[m]=mpc * c).
*

If I understand this correctly it means that any non-modifiable part

of an AI will sooner or later be the limiting factor for its growth.

This leads to a very large risk in the long run of a competing system

without a non-modifiable part to become more efficient/powerful than

the one with non-modifiable parts.

For me the conclusion is clear:

Any AI with a hard-coded law for "being nice","prevent murders" or any

other non-modifiable procedure will with great probability be overrun

by a system without those hardwritten rules.

**Next message:**Vladimir Nesov: "[sl4] Weakening morality"**Previous message:**John K Clark: "Re: [sl4] Uploads coming first would be good, right?."**In reply to:**Johnicholas Hines: "[sl4] simplistic models of capability growth"**Next in thread:**Johnicholas Hines: "Re: [sl4] simplistic models of capability growth"**Reply:**Johnicholas Hines: "Re: [sl4] simplistic models of capability growth"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ] [ attachment ]

*
This archive was generated by hypermail 2.1.5
: Wed Jul 17 2013 - 04:01:04 MDT
*