LessWrong 2.0 Reader

Some orgs did that and it generally didn't go well (eg Leverage Research). I think most people believe that totalizing jobs are bad for mental health and create bad epistemics and it's not worth it.

Working hard together with similarly minded people seems great. Never taking a break, and isolating yourself from the world, is not.

People working at startups usually get at least free weekends, and often have a partner at home who is not a member of the startup. If you never take a break, I suspect that you are optimizing for appearing to work hard, rather than for actually being productive.

I have read the "TESCREAL" paper recently, and wrote some thoughts about it in an ACX Open Thread.

It also gave me conspiracy theory vibes, as it tried too hard to connect together various groups and people that are parts of the sinister-sounding "TESCREAL" (including a table of individuals and organizations involved in various parts), trace their roots back to eugenicists (but also Plato and Aristotle), and warn about their wealth and influence.

It reminded me how some people in my country love to compile lists of people working at various non-profits to prove how this is all linked to Soros and how they are all servants of American propaganda trying to destroy our independence. Because apparently you cannot volunteer in a shelter for abandoned puppies without being a part of some larger sinister plot.

From the Dark Arts perspective, I think it would be useful to sigh and say "oh, this conspiracy theory again?" to signal that you consider the authors low-status. But then focus on the object-level objections.

The actual objection, from my perspective, is that the thing that connects the parts of the "TESCREAL" is simply "nerds who care, and think that technology is the answer". Some parts are more strongly related; if you believe in technological progress, then longtermism and transhumanism and extropianism and cosmism are more or less the same thing, the belief that in future, humans will overcome their current limitations using technology. That should not really come as huge a surprise for anyone.

The connection with EA is cherry-picking; yes, there are some longtermist projects, but most of it is stuff like curing malaria. But of course, you can't say that, if your agenda is to call them Nazis eugenicists.

And the connection with eugenicists is mostly "you know who else worried about the future of humanity?" (I find it difficult to think of a more appropriate response than "fuck you!") But also, speaking about intelligence is a taboo, which means that it is a taboo to worry about artificial intelligences becoming potentially smarter than humans. -- Here, I think a potential solution would be to push the authors towards making some object-level statements. Not just "people who say X are like Hitler eugenicists", but state your opinion clearly, whether it is "X" or "not X"; make a falsifiable statement.

But I think it is not too uncharitable to summarize the paper as "a conspiracy theory claiming that people who donate money to African charities that cure malaria are secretly eugenicists", because that is an important part of the "TESCR-EA-L" construct.

I'd say that the reason why the SpaceX cult/business can actually make working rockets is because they have rich feedback from reality when they try to design rockets, even at the pre-testing stage, because while it's not obvious to a layperson if a rocket does work, it is relatively easy to check the physics of whether a new rocket does work for an expert, meaning the checking of claims can be made legible, which is an enemy to cults in general.

More generally, I'd say the difference between a cult and a high-impact startup/business is whether they can get rich and reliable feedback from a source, and secondarily how legible their theory of impact/claims are.

Bigness alone doesn't cut it.

I don't know anything about the card. I haven't re-read the post, but I think the point I was making was "you haven't successfully argued that this is good cost-benefit", not "I claim that this is bad cost-benefit". Another possibility is that I was just pointing out that the specific quoted paragraph had an implied bad argument, but I didn't think it said much about the post overall.

(I think individual FB questions can toggle whether to show/hide predictions before you've made your own)

I think it should be hidden by default in the editor, with a user-side setting to show by default for all questions. 

Suppose one buys your thesis that most or all animals are conscious and feel intense pain. What is to be done ? Upload the shrimp ?

Cool project! Thanks for doing it and sharing, great to see more models with SAEs

interpretability research on proprietary LLMs that was quite popular this year and great research papers by Anthropic[1][2], OpenAI[3][4] and Google Deepmind

I run the Google DeepMind team, and just wanted to clarify that our work was not on proprietary closed weight models, but instead on Gemma 2, as were our open weight SAEs - Gemma 2 is about as open as llama imo. We try to use open models wherever possible for these general reasons of good scientific practice, ease of replicability, etc. Though we couldn't open source the data, and didn't go to the effort of open sourcing the code, so I don't think they can be considered true open source. OpenAI did most of their work on gpt2, and only did their large scale experiment on GPT4 I believe. All Anthropic work I'm aware of is on proprietary models, alas.

2. The style vaguely feels like something ChatGPT might right. Brightly polished, safe and stale.

It is definitely ChatGPT. There are a lot of things in the essay that make no sense the moment you stop and think about what is actually being said [LW · GW]. For example:

At its core, UBI is about ensuring that everyone has the financial resources to meet their basic needs.

Not "at its core". That is what UBI is.

For businesses, UBI provides a stable customer base...

A customer base for buying basic necessities, but not for anything above that, like a shiny new games console. And a customer base for basic necessities already exists. Broadly speaking (a glance at Wikipedia), in the developed world it falls about 10 to 20% short of being the entire population, and there are typically government programs of some sort to assist most of them.

...and a workforce

How does UBI provide a workforce? UBI pays people whether they work or not. That's what the U means. One of the motivations for UBI is a predicted lack of any useful employment for large numbers of people in the near future.

By investing in UBI, businesses can

How does a business "invest in UBI"? UBI is paid by the government out of taxes.

The beauty of UBI lies in its potential to align individual aspirations with collective progress. By ensuring that basic needs are met, we free people to contribute their skills and energy to areas where they’re most needed

People will already pay people to do the work that they need done. Is it envisaged that under UBI, people will joyfully "contribute their skills and energy" without pay, at whatever work someone has judged to be "needed"? I don't know, but the more I look at this passage the more the apparent meaning drains out of it. There is nothing here but hurrah words. There is nothing in the whole essay.

The thing is that, if you roll a 6 and then a non-6, in an "A" sequence you're likely to just die due to rolling an odd number before you succeed in getting the double 6, and thus exclude the sequence from the surviving set; whereas in a "B" sequence there's a much higher chance you'll roll a 6 before dying, and thus include this longer "sequence of 3+ rolls" in the set.

Yes! This kind of kills the "paradox". Its approaching an apples and oranges comparison.

Surviving sequences with n=100 rolls (for illustrative purposes)

[6, 6]
[6, 6]
[2, 6, 6]
[6, 6]
[2, 6, 6]
[6, 6]
Estimate for A: 2.333
[6, 6]
[4, 4, 6, 2, 2, 6]
[6, 6]
[6, 2, 4, 4, 6]
[6, 4, 6]
[4, 4, 6, 4, 6]
[6, 6]
[6, 6]
Estimate for B: 3.375

if you rephrase

$A$: The probability that you roll a fair die until you roll two $6\text{s}$ in a row, given that all rolls were even.

$B$: The probability that you roll a fair die until you roll two non-consecutive $6s$ (not necessarily in a row), given that all rolls were even.

This changes the code to:

A_estimate = num_sequences_without_odds/n

B_estimate = num_sequences_without_odds/n

And the result (n=100000) 

Estimate for A: 0.045
Estimate for B: 0.062

I guess this is what most people where thinking when reading the problem, i.e., its a bigger chance of getting two non consecutive 6s. But by the wording (see above) of the "paradox" it gives more rolls on average for the surviving sequences, but you on the other hand have more surviving sequences hence higher probability.

Sometimes the thing that seems like zero-sum between two players actually has a third player, let's call them "audience" or "environment", and the payout is different when you include those. Two people trying to win a tennis match provide entertainment for the audience. Also, in short term, one of the players wins and the other one loses, but in long term, both have practiced their skills and had some healthy exercise.

Status seeking is immoral when it comes to conflict with doing the right thing. Sometimes that means cheating to appear better than you actually are. Sometimes it means generating negative externalities.

But in a healthy environment, social status can be a way to recognize and reward doing the right thing.