LessWrong 2.0 Reader

I definitely agree that specific examples would make the argument much stronger. At least, it would allow me to understand what kind of "false alarms" are we talking about here: is it mere tech hypes (such as cold fusion), or specifically humanity-destroying events (such as nuclear war)?

I think we didn't have so many things that threatened to destroy humanity. Maybe it's just my ignorance speaking, but the nuclear war is the only example that comes to my mind. (Global warming, although possibly a great disaster, is alone not an extinction threat to entire humanity.) And mere tech hypes that didn't threaten to destroy humanity don't seem like a relevant category for the AI danger.

Perhaps more importantly, with things like LK99 or cold fusion, the only source of hype was "people enthusiastically writing papers". With AI, the situation is more like "anyone can use (for free, if it's only a few times a day) a technology that would be considered sci-fi five years ago". Like, the controversy is about how far and fast it will get, but there is no doubt that it is already here... and even if somehow magically the state of AI would never improve beyond where it is today, we would still have a few more years of social impact at more people would learn to use it and find new ways how to use it.

EDIT: By "sci-fi" I mean, imagine creating a robotic head that uses speech recognition and synthesis to communicate with humans, uploading the latest LLM into it, and sending it by a time machine five or ten years into the past. Or rather, sending thousands of such robotic heads. People would be totally scared (not just because of the time travel). And finding out that the robotic heads often hallucinate would only calm them down a little.

Linda Linsefors of the Center for Applied Rationality

Hi, thanks for the mention.

But I'd like to point out that I never worked for CFAR in any capacity. I have attended two CFAR workshops. I think that calling me "of the Center for Applied Rationality" is very misleading, and I'd prefer it if you remove that part, or possibly re-phrase it.

nothing short of death can stop me from trying to do good.

the world could destroy or corrupt EA, but i'd remain an altruist.

it could imprison me, but i'd stay focused on alignment, as long as i could communicate to at least one on the outside.

even if it tried to kill me, i'd continue in the paths through time where i survived.

How about this: trauma is a set of one or more habits that

1. was adaptive at one time but isn't anymore and
2. is hurting you or likely to hurt you in some way.

Something like a safety case for automated safety research (but I'm biased)

No answer here, but I would like to see some book for teenagers that would be like "a list of serious problems that need to be solved", compiled by reasonable people, so that it wouldn't be completely one-sided. This might make the world a better place, if everyone could pick a topic that resonates with them most. (Rather than the default, which is "choose your group, and try to destroy the outgroup by any means possible".)

The hypothetical most popular president (from the perspective of the entire population) would lose in the primaries. Their own party would never nominate them, because they would seem like a sell-out to them.

Imagine the following 3 candidates:

A -- strongly against illegal immigrants, and quite racist against the legal ones

B -- suggests to stop illegal immigration, but make the legal immigration much easier

C -- wants to make immigration easier; refuses to debate illegal immigration because "no one is illegal"

A would win the Republican primaries, C would win Democratic primaries. B would lose both.

For most people, B would be preferable to either A or C. But they won't get to make that choice. They will have to choose between A and C.

.

Let's make it more complicated, and split the candidate B into two similar candidates: B-R and B-D. Both B-R and B-D have the same position on immigration, but on different topics, B-R leans slightly Republican, and B-D leans slightly Democrat.

A hypothetical rational Republican might say "let's nominate B-R for our party, because they are most likely to get elected, and at least they agree with us on many other issues -- the 100% chance of B-R winning is preferable to a 50% chance of A and a 50% chance of C". A hypothetical rational Democrat might similarly prefer a 100% chance of B-D over a 50% chance of A and a 50% chance of C.

(And basically, this is what the median voter theorem suggests: that the election will ultimately be between B-R and B-D, rather than between A and C.)

But in a situation with primaries, B-R will lose to A, and B-D will lose to C.

.

I suspect that to a smaller degree this might be a problem with political parties in general, even if without primaries it is probably much smaller. Individuals need allies to win, and people like B-R and B-D won't find many enthusiastic allies in their respective parties.

I think you are right. We will not observe QI agents and it is a bad policy to recommend it as I will end in empty world soon. Now caveats. My measure declines because of branching anyway very quickly, so no problem. There is an idea of civilization-level quantum suicide by Paul Almond. In that case, the whole civilization performs QI coin trick, and no problem with empty world - but can explain Fermi paradox. QI make sense from first-person perspective, but not from third.

Democrats lost by sufficient margins, and sufficiently broadly, that one can make the argument that any pet cause is a responsible or contributing factor. But that seems like entirely the wrong level of analysis to me. See this FT article called "Democrats join 2024’s graveyard of incumbents", which includes this striking graph:

So global issues like inflation and immigration seem like much better explanatory factors to me, rather than things like the Gaza conflict which IIRC never made the top 10 in any issue polling I saw.

(The article may be paywalled; I got the unlocked version by searching Bing for "ft every governing party facing election in a developed country".)

I feel that the goal here is to reproduce the effects of choosing a president by popular vote without abolishing the electoral college. Wouldn't an electoral reform to abolish the electoral college be a more realistic goal?