LessWrong 2.0 Reader

In fact, this mindset gave me burnout earlier this year.

I relate pretty strongly to this. I think almost all junior researchers are incentivised to 'paper grind' for longer than is correct. That said, I do think there are pretty strong returns to having one good paper for credibility reasons; it signals that you are capable of doing AI safety research, and thus makes it easier to apply for subsequent opportunities.

Over the past 6 months I've dropped the paper grind mindset and am much happier for this. Notably, were it not for short term grants where needing to visibly make progress is important, I would have made this update sooner. A related take that I have is that if you have the flexibility to do so, front-loading learning time seems good. See here for a related take by Rohin. Making progress on hard problems requires understanding things deeply, in a way which making progress on easy problems that you could complete during e.g. MATS might not.

If it would be only true in the case of calorie restriction, why don't we have better studies about the effects of salt?

People like to eat together with other people. They go together to restaurants to eat shared meals. They have family dinners. 

You might want to stop using the honey extension. Here are some shady things they do, beyond the usual:

1. Steal affiliate marketing revenue from influencers (who they also often sponsor), by replacing the genuine affiliate referral cookie with their affiliate referral cookie.
2. Deceive customers by deliberately withholding the best coupon codes, while claiming they have found the best coupon codes on the internet; partner businesses control which coupon codes honey shows consumers.

I'm not sure if any country has successfully been able to withstand the pressures of the processed food industry once it has entered their country. At least, I couldn't find any examples in my research.

The only countries that potentially could make quick resolutions are those that have high levels of state power over personal freedoms, like China. But so far that hasn't happened yet.

Places like the US will likely continue to suffer from chronic disease in the short-to-medium term (~10-50 years) due to its emphasis on personal freedom to be able to deteriorate one's health. 

I'm not sure "we're super dumb as [a] society" so much as we're gridlocked by special interest groups. We were able to take action against the tobacco industry because nobody has to smoke. But everyone's gotta eat.

The main problem here is that this approach doesn't solve alignment, but merely shifts it to another system. We know that human organizational systems also suffer from misalignment - they are intrinsically misaligned. Here are several types of human organizational misalignment:

• Dictatorship: exhibits non-corrigibility, with power becoming a convergent goal
• Goodharting: manifests the same way as in AI systems
• Corruption: acts as internal wireheading
• Absurd projects (pyramids, genocide): parallel AI's paperclip maximization
• Hansonian organizational rot: mirrors error accumulation in AI systems
• Aggression: parallels an AI's drive to dominate the world

All previous attempts to create a government without these issues have failed (Musk's DOGE will likely be another such attempt).

Furthermore, this approach doesn't prevent others from creating self-improving paperclippers.

"If you already know that an adverse event is highly likely for your specific circumstances, then it is likely that the insurer will refuse to pay out for not disclosing "material information" - a breach of contract."

Having worked in insurance, that's not what the companies usually do. Denying explicitly for clear but legally hard to defend reasons, especially those which a jury would likely rule against, isn't a good way to reduce costs and losses. (They usually will just say no and wait to see if you bother following up. Anyone determined enough to push to get a reasonable claim is gonna be cheaper to pay out for than to fight.)

Given so many shared premises, it's puzzling to me why Egan seems to bear so much antipathy towards "us"

This is a fairly well-documented phenomenon: the narcissism of small differences. 

Also:

the OpenPhil people and the MIRI people and the Vassarites and ... &c. are all totally different and in fact hate each other's guts

is clearly an instance of the same phenomenon.

But why do we care more about statistical relationships between physical humans and dogs than statistical relationships between the word "human" and the word "dog" as characters on your screen?

An overly-cute but not completely wrong answer: because I care about whether AI kills all the physical humans, not whether something somewhere writes the string "kill all the humans". My terminal-ish values are mostly over the physical stuff.

I think the point you're trying to make is roughly "well, it's all pretty entangled with the physical stuff anyway, so why favor one medium or another? Instrumentally, either suffices.". And the point I'm trying to make in response is "it matters a lot how complicated the relationship is between the medium and the physical stuff, because terminally it's the physical we care about, so instrumentally stuff that's more simply related to the physical stuff is a lot more useful to understand.".

Bit of a tangent, but topical: I don't think language models are individual minds. My current max likelihood mental model is that part of the base level suggestibility is because the character level is highly uncertain, due to being a model of the characters of many humans. I agree that the character level appears to have some properties of personhood. Language models are clearly some forms of morally relevant, most obviously I see them as a reanimation of a blend of other minds, but it's not clear what internal phenomena are negative for the reanimated mind. The equivalence to slavery seems to me better expressed by saying they approximately reanimated mind-defining data without the consent of the minds being reanimated; the way people express this is normally to say things like "stolen data".

Has someone made an ebook that I can easily download onto my kindle?

I'm unclear if a good ebook should include all the pictures from the original version.