LessWrong 2.0 Reader

I'd say that the reason why the SpaceX cult/business can actually make working rockets is because they have rich feedback from reality when they try to design rockets, even at the pre-testing stage, because while it's not obvious to a layperson if a rocket does work, it is relatively easy to check the physics of whether a new rocket does work for an expert, meaning the checking of claims can be made legible, which is an enemy to cults in general.

More generally, I'd say the difference between a cult and a high-impact startup/business is whether they can get rich and reliable feedback from a source, and secondarily how legible their theory of impact/claims are.

Bigness alone doesn't cut it.

I don't know anything about the card. I haven't re-read the post, but I think the point I was making was "you haven't successfully argued that this is good cost-benefit", not "I claim that this is bad cost-benefit". Another possibility is that I was just pointing out that the specific quoted paragraph had an implied bad argument, but I didn't think it said much about the post overall.

(I think individual FB questions can toggle whether to show/hide predictions before you've made your own)

I think it should be hidden by default in the editor, with a user-side setting to show by default for all questions. 

Suppose one buys your thesis that most or all animals are conscious and feel intense pain. What is to be done ? Upload the shrimp ?

Cool project! Thanks for doing it and sharing, great to see more models with SAEs

interpretability research on proprietary LLMs that was quite popular this year and great research papers by Anthropic[1][2], OpenAI[3][4] and Google Deepmind

I run the Google DeepMind team, and just wanted to clarify that our work was not on proprietary closed weight models, but instead on Gemma 2, as were our open weight SAEs - Gemma 2 is about as open as llama imo. We try to use open models wherever possible for these general reasons of good scientific practice, ease of replicability, etc. Though we couldn't open source the data, and didn't go to the effort of open sourcing the code, so I don't think they can be considered true open source. OpenAI did most of their work on gpt2, and only did their large scale experiment on GPT4 I believe. All Anthropic work I'm aware of is on proprietary models, alas.

2. The style vaguely feels like something ChatGPT might right. Brightly polished, safe and stale.

It is definitely ChatGPT. There are a lot of things in the essay that make no sense the moment you stop and think about what is actually being said [LW · GW]. For example:

At its core, UBI is about ensuring that everyone has the financial resources to meet their basic needs.

Not "at its core". That is what UBI is.

For businesses, UBI provides a stable customer base...

A customer base for buying basic necessities, but not for anything above that, like a shiny new games console. And a customer base for basic necessities already exists. Broadly speaking (a glance at Wikipedia), in the developed world it falls about 10 to 20% short of being the entire population, and there are typically government programs of some sort to assist most of them.

...and a workforce

How does UBI provide a workforce? UBI pays people whether they work or not. That's what the U means. One of the motivations for UBI is a predicted lack of any useful employment for large numbers of people in the near future.

By investing in UBI, businesses can

How does a business "invest in UBI"? UBI is paid by the government out of taxes.

The beauty of UBI lies in its potential to align individual aspirations with collective progress. By ensuring that basic needs are met, we free people to contribute their skills and energy to areas where they’re most needed

People will already pay people to do the work that they need done. Is it envisaged that under UBI, people will joyfully "contribute their skills and energy" without pay, at whatever work someone has judged to be "needed"? I don't know, but the more I look at this passage the more the apparent meaning drains out of it. There is nothing here but hurrah words. There is nothing in the whole essay.

The thing is that, if you roll a 6 and then a non-6, in an "A" sequence you're likely to just die due to rolling an odd number before you succeed in getting the double 6, and thus exclude the sequence from the surviving set; whereas in a "B" sequence there's a much higher chance you'll roll a 6 before dying, and thus include this longer "sequence of 3+ rolls" in the set.

Yes! This kind of kills the "paradox". Its approaching an apples and oranges comparison.

Surviving sequences with n=100 rolls (for illustrative purposes)

[6, 6]
[6, 6]
[2, 6, 6]
[6, 6]
[2, 6, 6]
[6, 6]
Estimate for A: 2.333
[6, 6]
[4, 4, 6, 2, 2, 6]
[6, 6]
[6, 2, 4, 4, 6]
[6, 4, 6]
[4, 4, 6, 4, 6]
[6, 6]
[6, 6]
Estimate for B: 3.375

if you rephrase

$A$: The probability that you roll a fair die until you roll two $6\text{s}$ in a row, given that all rolls were even.

$B$: The probability that you roll a fair die until you roll two non-consecutive $6s$ (not necessarily in a row), given that all rolls were even.

This changes the code to:

A_estimate = num_sequences_without_odds/n

B_estimate = num_sequences_without_odds/n

And the result (n=100000) 

Estimate for A: 0.045
Estimate for B: 0.062

I guess this is what most people where thinking when reading the problem, i.e., its a bigger chance of getting two non consecutive 6s. But by the wording (see above) of the "paradox" it gives more rolls on average for the surviving sequences, but you on the other hand have more surviving sequences hence higher probability.

Sometimes the thing that seems like zero-sum between two players actually has a third player, let's call them "audience" or "environment", and the payout is different when you include those. Two people trying to win a tennis match provide entertainment for the audience. Also, in short term, one of the players wins and the other one loses, but in long term, both have practiced their skills and had some healthy exercise.

Status seeking is immoral when it comes to conflict with doing the right thing. Sometimes that means cheating to appear better than you actually are. Sometimes it means generating negative externalities.

But in a healthy environment, social status can be a way to recognize and reward doing the right thing.

The abstract descriptions are sometimes leaky. Recently I had a short training on "how to solve problems", which assumed discrete steps, something like "define the problem", "collect facts", "find out causes", "propose a fix".

Some people seem impressed by seeing four or five bullet points that in theory would allow them to solve any problem, if they only apply them in the right sequence. To me, it seems like the real world does not follow this "waterfall" model.

For example, collecting the facts. Which facts? About what? How much certainty do you need for those facts? If you really tried to collect all facts about a nontrivial system, it would take much more time than you usually have. So you go with some assumptions, like "these are the things that usually break, let's check them first" and "these are the things that are cheap to check, so let's check those, too". And you solve half of the problems like this, quickly. For the other half, you take a step back, check a few more things, etc. And when the things really get complicated, then you start going step by step, verifying one step at a time. But it would be a mistake to say that you wasted your time by trying to solve the problem before you had all the facts. You did something that works well on average, it's just that problem we had today was an outlier.

(I am not saying that this is completely useful. Sometimes people make the mistake of going in circles without taking a step back and checking their premises, so it is good to remind them to collect more data. But you need to be adaptive about that, which is something that in my case the high-level description abstracted away.)

Indeed