Pulling the trigger, part 2

On the importance of collaboration

In my original post on this topic, I opined that AI developers should in general regard each other as collaborators, not as competitors. I'd like to expand on that claim, and to demonstrate that even developers who are solely motivated by selfish considerations will find that the best way to achieve their goals is to collaborate with their competitors in a seemingly selfless manner.

For the sake of discussion, let's consider two possible approaches to the singularity:

A) Do everything you ethically can to ensure that yours is the AI project that triggers the singularity.

B) Do everything you can to maximize the odds of the singularity being beneficial, regardless of whose project ends up triggering it.

Obviously, strategy B is preferable from an altruistic perspective. I claim, however, that strategy B is preferable even if you have a strictly selfish utility function. The argument goes like this:

There will be an interim period of time between the initial deployment of viable seed AI technology and the onset of a full-blown singularity. Depending on where you stand on the "soft takeoff" vs "hard takeoff" debate, that period of time will be somewhere between a few hours and a few decades. During that interim period, being the owner of the "winning" AI confers significant selfish benefits (fame, glory, wealth beyond measure, etc.) Let's refer to the duration of the interim period as T1, and the utility per unit time of winning the AI race as U1.

Once the singularity occurs, however, it doesn't matter who launched it. Assuming that the singularity is beneficial, the benefits to all of us will be immense, and will endure for a near infinite period of time. Let's refer to the duration of the post-singularity period as T2, and the utility per unit time of experiencing a beneficial singularity as U2.

The benefit of strategy A is that it increases your odds of winning the AI race by some amount which we'll call P(A). Similarly, the benefit of strategy B is that it increases the odds of the singularity being beneficial by some amount which we'll call P(B). So, the total utility of strategy A is

P(A) * T1 * U1

Similarly, the total utility of strategy B is

P(B) * T2 * U2

So, strategy B is preferable if

P(B) * T2 * U2 > P(A) * T1 * U1

Knowing that U2 >= U1, we can say conservatively that strategy B is preferable if

P(B) / P(A) > T1 / T2

Given that T1 <= 50 years and T2 >= 1 billion years, we can say conservatively that strategy B is preferable if

P(B) / P(A) > 1 / 20,000,000

The bottom line is this: even if you're a selfish bastard, you should focus your efforts exclusively on maximizing the odds of a beneficial singularity, and not on ensuring that your own AI wins the singularity race. This conclusion has some specific practical implications that I'll consider next time.