LessWrong 2.0 Reader

Cool project! Thanks for doing it and sharing, great to see more models with SAEs

interpretability research on proprietary LLMs that was quite popular this year and great research papers by Anthropic[1][2], OpenAI[3][4] and Google Deepmind

I run the Google DeepMind team, and just wanted to clarify that our work was not on proprietary models, but instead on Gemma 2, as were our open weight SAEs. We try to use open source models wherever possible for these general reasons of good scientific practice, ease of replicability, etc. Though we couldn't open source the data, and didn't go to the effort of open sourcing the code, so I don't think they can be considered true open source. OpenAI did most of their work on gpt2, and only did their large scale experiment on GPT4 I believe. All Anthropic work I'm aware of is on proprietary models, alas.

2. The style vaguely feels like something ChatGPT might right. Brightly polished, safe and stale.

It is definitely ChatGPT. There are a lot of things in the essay that make no sense the moment you stop and think about what is actually being said [LW · GW]. For example:

At its core, UBI is about ensuring that everyone has the financial resources to meet their basic needs.

Not "at its core". That is precisely what UBI is.

For businesses, UBI provides a stable customer base...

A customer base for buying basic necessities, but not for anything above that, like a shiny new games console. And a customer base for basic necessities already exists. Broadly speaking (a glance at Wikipedia), in the developed world it falls about 10 to 20% short of being the entire population.

...and a workforce

How does UBI provide a workforce? UBI pays people whether they work or not. That's what the U means. One of the motivations for UBI is a predicted lack of any useful employment for large numbers of people in the near future.

By investing in UBI, businesses can

How does a business "invest in UBI"? UBI is paid by the government out of taxes.

The beauty of UBI lies in its potential to align individual aspirations with collective progress. By ensuring that basic needs are met, we free people to contribute their skills and energy to areas where they’re most needed

People will already pay people to do the work that they need done. Is it envisaged that under UBI, people will joyfully "contribute their skills and energy" without pay, at whatever work someone has judged to be "needed"? I don't know, but the more I look at this passage the more the apparent meaning drains out of it. There is nothing here but hurrah words. There is nothing in the whole essay.

The thing is that, if you roll a 6 and then a non-6, in an "A" sequence you're likely to just die due to rolling an odd number before you succeed in getting the double 6, and thus exclude the sequence from the surviving set; whereas in a "B" sequence there's a much higher chance you'll roll a 6 before dying, and thus include this longer "sequence of 3+ rolls" in the set.

Yes! This kind of kills the "paradox". Its approaching an apples and oranges comparison.

Surviving sequences with n=100 rolls (for illustrative purposes)

[6, 6]
[6, 6]
[2, 6, 6]
[6, 6]
[2, 6, 6]
[6, 6]
Estimate for A: 2.333
[6, 6]
[4, 4, 6, 2, 2, 6]
[6, 6]
[6, 2, 4, 4, 6]
[6, 4, 6]
[4, 4, 6, 4, 6]
[6, 6]
[6, 6]
Estimate for B: 3.375

if you rephrase

$A$: The probability that you roll a fair die until you roll two $6\text{s}$ in a row, given that all rolls were even.

$B$: The probability that you roll a fair die until you roll two non-consecutive $6s$ (not necessarily in a row), given that all rolls were even.

This changes the code to:

A_estimate = num_sequences_without_odds/n

B_estimate = num_sequences_without_odds/n

And the result (n=100000) 

Estimate for A: 0.045
Estimate for B: 0.062

I guess this is what most people where thinking when reading the problem, i.e., its a bigger chance of getting two non consecutive 6s. But by the wording (see above) of the "paradox" it gives more rolls on average for the surviving sequences, but you on the other hand have more surviving sequences hence higher probability.

Sometimes the thing that seems like zero-sum between two players actually has a third player, let's call them "audience" or "environment", and the payout is different when you include those. Two people trying to win a tennis match provide entertainment for the audience. Also, in short term, one of the players wins and the other one loses, but in long term, both have practiced their skills and had some healthy exercise.

Status seeking is immoral when it comes to conflict with doing the right thing. Sometimes that means cheating to appear better than you actually are. Sometimes it means generating negative externalities.

But in a healthy environment, social status can be a way to recognize and reward doing the right thing.

The abstract descriptions are sometimes leaky. Recently I had a short training on "how to solve problems", which assumed discrete steps, something like "define the problem", "collect facts", "find out causes", "propose a fix".

Some people seem impressed by seeing four or five bullet points that in theory would allow them to solve any problem, if they only apply them in the right sequence. To me, it seems like the real world does not follow this "waterfall" model.

For example, collecting the facts. Which facts? About what? How much certainty do you need for those facts? If you really tried to collect all facts about a nontrivial system, it would take much more time than you usually have. So you go with some assumptions, like "these are the things that usually break, let's check them first" and "these are the things that are cheap to check, so let's check those, too". And you solve half of the problems like this, quickly. For the other half, you take a step back, check a few more things, etc. And when the things really get complicated, then you start going step by step, verifying one step at a time. But it would be a mistake to say that you wasted your time by trying to solve the problem before you had all the facts. You did something that works well on average, it's just that problem we had today was an outlier.

(I am not saying that this is completely useful. Sometimes people make the mistake of going in circles without taking a step back and checking their premises, so it is good to remind them to collect more data. But you need to be adaptive about that, which is something that in my case the high-level description abstracted away.)

Indeed

In a world where AI progress has wildly accelerated chip manufacture

This world?

My cousin had a small restaurant, and one of the reasons he gave up was complying with all kinds of regulations. There was always some small detail that was wrong, and he typically did not get fined, he was just told to fix it before the next review. But it took a lot of his time. It was usually some stupid thing, such as painting the walls incorrectly, or using a wrong kind of screw on the roof, i.e. not even related to the food.

Also, one of the reasons it is difficult to build small kindergartens (like for 10 kids total) is that they are required to have a kitchen, which is required to have a certain area and inventory, which at this scale becomes the most expensive part of the kindergarten. But if you invite guests to your home, cooking for 10 people in your kitchen is easy, even if would completely fail the kindergarten regulations.

(Not in USA. Your local regulations may differ.)

I have time-space synesthesia, so I actually picture some times as being literally farther away than others.

I visualize the months of the year in a disc slanted away from me, kind of like a clock with New Years being at 6pm, and visualize years on a number line.

My guess as to why this got down-voted:
1) This reads like a manifesto, and not an argument. It reads like an aspirational poster, and not a plan. It feels like marketing, and not communication. 
2) The style vaguely feels like something ChatGPT might right. Brightly polished, safe and stale.
3) This post doesn't have any clear connection to making people less-wrong or reducing x-risks. 

3) wouldn't have been much of an issue if not for 1 and 2. And 1 is an issue because, for the most part, LW has an aversion to "PR [LW · GW]". 2 is an issue because ChatGPT is now a thing so styles of writing which are like ChatGPT's are viewed as likely to have been written by ChatGPT. This is an issue because texts written by ChatGPT often have little thought put into them, are unlikely to contain much that's novel, and frequently have errors. 

What kind of post could you have written which would have been better received? I'll give some examples. 

1) A concrete proposal for UBI that you thought was under-valued
2) An argument addressing some problems people have with UBI (e.g. who pays for all of it? After UBI is implemented and society reaches an equilibrium, won't rents-seeking systems just suck up all the UBI money leaving people no better off than before?). 
3) Or a post which was explicit about wanting to get people interested in UBI, and asked for feedback on potential draft messages. 

In general, if you had informed people of something you genuinely believe, or told them about something you have tried and found useful, or asked sincere questions, then I think you'd have got a better reception.