LessWrong 2.0 Reader

@jefftk [LW · GW] comments on the HN thread on this: 

How many people would, if they suddenly died, be reported as a "Boeing whistleblower"? The lower this number is, the more surprising the death.

Another HN commenter says (in a different thread): 

It’s a nice little math problem.

Let’s say both of the whistleblowers were age 50. The probability of a 50 year old man dying in a year is 0.6%. So the probability of 2 or more of them dying in a year is 1 - (the probability of exactly zero dying in a year + the probability of exactly one dying in a year). 1 - (A+B).

A is (1-0.006)^N. B is 0.006N(1-0.006)^(N-1). At 60 A is about 70% and B is about 25% making it statistically insignificant.

But they died in the same 2 month period, so that 0.006 should be 0.001. If you rerun the same calculation, it’s 356.

So... does anyone have any takes on the two Boeing whistleblowers who died under somewhat suspicious circumstances? I haven't followed this in detail, and my guess is it is basically just random chance, but it sure would be a huge deal if a publicly traded company now was performing assassinations of U.S. citizens. 

Curious whether anyone has looked into this, or has thought much about baseline risk of assassinations or other forms of violence from economic actors.

Yes, such as: "Anything worth doing is worth doing [well/poorly]" needs more qualifiers to be meaningful. As it is the opposite advice can often be just as useful. I.E. not very.

Better V1: "The cost/utility ratio of beneficial actions at minimum cost are often less favorable than they would be with greater investment."

Better V2: "If an action is beneficial, a flawed attempt may be preferable to none at all."

However, these are too wordy to be pithy and in pop culture transmission accuracy is generally sacrificed in favor of catchiness.

One operationalization is "these AIs are capable of speeding up ML R&D by 30x with less than a 2x increase in marginal costs".

As in, if you have a team doing ML research, you can make them 30x faster with only <2x increase in cost by going from not using your powerful AIs to using them.

With these caveats:

• The speed up is relative to the current status quo as of GPT-4.
• The speed up is ignoring the "speed up" of "having better experiments to do due to access to better models" (so e.g., they would complete a fixed research task faster).
• By "capable" of speeding things up this much, I mean that if AIs "wanted" to speed up this task and if we didn't have any safety precautions slowing things down, we could get these speedups. (Of course, AIs might actively and successfully slow down certain types of research and we might have burdensome safety precautions.)
• The 2x increase in marginal cost is ignoring potential inflation in the cost of compute (FLOP/$) and inflation in the cost of wages of ML researchers. Otherwise, I'm uncertain how exactly to model the situation. Maybe increase in wages and decrease in FLOP/$ cancel out? Idk.
• It might be important that the speed up is amortized over a longer duration like 6 months to 1 year.

I'm uncertain what the economic impact of such systems will look like. I could imagine either massive (GDP has already grown >4x due to the total effects of AI) or only moderate (AIs haven't yet been that widely deployed due to inference availability issues, so actual production hasn't increased that much due to AI (<10%), though markets are pricing in AI being a really, really big deal).

So, it's hard for me to predict the immediate impact on world GDP. After adaptation and broad deployment, systems of this level would likely have a massive effect on GDP.

Do you want to make a demo with dspy+gpt4 api+fast downward?

A lot of this depends on where you draw the line between 'rationality' and 'science' or 'economics' and 'philosophy' or so on. As well, given that 'rationality' is doing the best you can given the constraints you're under, it seems likely that many historical figures were 'rational' even if they weren't clear precursors to the modern rationalist cluster.

For example, I think Xunzi (~3rd century BCE) definitely counts; check out Undoing Fixation in particular. [His students Li Si and Han Fei are also interesting in this regard, but I haven't found something by them yet that makes them clearly stand out as rationalists. Also, like JenniferRM points out, they had a troubled legacy somewhat similar to Alexander's.]

Some people count Mozi as the 'first effective altruist' in a way that seems similar.

People point to Francis Bacon as the originator of empiricism; you can read his main work here on LW [LW · GW]. While influential in English-language thought, I think he is anticipated by al-Haytham and Ibn Sina.

LaPlace is primarily famous as a mathematician and scientist, but I think he was important in the development of math underpinning modern rationality, and likely counts.

Benjamin Franklin seems relevant in a handful of ways; his autobiography is probably the best place to start reading.

Alfred Korzybski is almost exactly a hundred years older than Yudkowsky, and is the closest I'm aware of to rationality-as-it-is-now. You can see a discussion of sources between then and now in Rationalism Before The Sequences [LW · GW].

Ohhhhh!

great, I bet my whole career and the future of the world on the premise that AI agents should be based on markets because markets are aligned and friendly, then Boeing just goes and kills two whistleblowers.

Early: That comes from AIs that are just powerful enough to be extremely useful and dangerous-by-default (i.e. these AIs aren’t wildly superhuman).

Can you be more clearer this point? To operationalize this, I propose the following question: what is the fraction of world GDP you expect will be attributable to AI at the time we have these risky AIs that you are interested in? 

For example, are you worried about AIs that will arise when AI is 1-10% of the economy, or more like 50%? 90%?

It suffices to show that the Smith lotteries that the above result establishes are the only lotteries that can be part of maximal lottery-lotteries are also subject to the partition-of-unity condition.

I fail to understand this sentence. Here are some questions about this sentence:

• what are Smith lotteries? Ctrl+f only finds lottery-Smith lottery-lotteries, do you mean these? Or do you mean lotteries that are smith?

• which result do you mean by "above result"?

• What does it mean for a lottery to be part of maximal lottery-lotteries?

• does "also subject to the partition-of-unity" refer to the smith lotteries or to the lotteries that are part of maximal lottery-lotteries? (it also feels like there is a word missing somewhere)

• Why would this suffice?

• Is this part also supposed to imply the existence of maximal lottery-lotteries? If so, why?