LessWrong 2.0 Reader

As someone who used to be fully sequence thinking-oriented and gradually came round to the cluster thinking view, I think it's useful to quote from that post of Holden's on when it's best to use which type of thinking:

I see sequence thinking as being highly useful for idea generation, brainstorming, reflection, and discussion, due to the way in which it makes assumptions explicit, allows extreme factors to carry extreme weight and generate surprising conclusions, and resists “regression to normality.” 

However, I see cluster thinking as superior in its tendency to reach good conclusions about which action (from a given set of options) should be taken. ...

Note that this distinction is not the same as the distinction between explicit expected value and holistic-intuition-based decision-making. Both of the thought processes above involve expected-value calculations; the two thought processes consider all the same factors; but they take different approaches to weighing them against each other. Specifically:

• Sequence thinking considers each parameter independently and doesn’t do any form of “sandboxing.” So it is much easier for one very large number to dominate the entire calculation even after one makes adjustments for e.g. expert opinion and other “outside views”...
• The two have very different approaches to what some call Knightian uncertainty (also sometimes called “model uncertainty” or “unknown unknowns”): the possibility that one’s model of the world is making fundamental mistakes and missing key parameters entirely... 

Also this:

Cluster thinking is more similar to empirically effective prediction methods

Sequence thinking presumes a particular framework for thinking about the consequences of one’s actions. It may incorporate many considerations, but all are translated into a single language, a single mental model, and in some sense a single “formula.” I believe this is at odds with how successful prediction systems operate, whether in finance, software, or domains such as political forecasting; such systems generally combine the predictions of multiple models in ways that purposefully avoid letting any one model (especially a low-certainty one) carry too much weight when it contradicts the others.

the first hybrid reasoning model on the market

Can somebody TLDR what they mean by "hybrid"? A naive interpretation would be "it can CoT-reason but it can also not-CoT-reason" but my understanding was that o1 and r1 do that already.

self-interest is often aligned with expanding your self boundaries to include others

Parenting strategies for blurring your kid's (or AI's) self-other boundaries:

1. Love. Love the kid. Give it a part of you. In return it will do the same.
2. Patience. Appreciate how the kid chooses to spend undirected time. Encourage the kid learn to navigate the world themselves at their own speed.
3. Stories. Give kid tools for empathy by teaching them to read, buying them a camera, or reciprocating their meanness/kindness.
4. Groups. Help kid enter collaborative playful spaces where they make and participate in games larger than themselves, eg sports teams, improv groups, pillow forts at sleepovers, etc.
5. Creation. Give them the materials/support to express themselves in media which last. Paintings, writing, sayings, clubs, tree-houses, songs, games, apps, characters, companies.

Epistemic status: riffing, speculation. Rock of salt: I don't yet have kids.

Pre-training, finetuning and RL are all types of training. But sure, expand 'train' to 'create' in order to include anything else like scaffolding. The point is it's not what you do in response to the outputs of the system, it's what the system tries to do.

I like the original title [NSFW] The Subspace Jhana better too. I wrote the entire post with that title in mind. This version loses the original's elegance.

Well please do derive it then, because to me it seems you just focused on one aspect and then concluded that that aspect definitely is the correct answer. 

If the goal was to reward the best and the brightest, then why does china make some of them disappear from time to time? Why reeducate the odd billionaire who misbehaves? The idea was to get him in power because he knows better and generates riches, no?

On giving away stuff for 'free': what would be good examples in your opinion? Steel? Silicon or finished solar cells? Electric cars and batteries? Masks useful during pandemics?

Seriously?

I agree with you on the network effects and winner takes all mechanics. But to me that is not related at all to exports and their subsidies. Just making good stuff at a reasonable price is enough. If desired, production can be subsidized, sure, but that has positive effects in your country as well. Chinese people own lots of real estate, top of the line electronics and electric cars, more so than we do. 

For what it's worth, I still prefer the original title, even after seeing the rationale for changing it. Oh well.

They cash in politically. 

Imagine some middle eastern oil and money rich country going abroad to help develop a subsaharan economy to help them extract their resources. Imagine china doing the same. Who will have more success?

Our cruxes is whether the amount of investment to build one has a positive expected return on investment, breaking down into

1. If you could populate such a city
2. Whether this is a "try everything regardless of cost" issue, given that a replacent is being developed for other reasons.

I suggest focusing on 1, as it's pretty fundamental to your idea and easier to get traction on