LessWrong 2.0 Reader

https://intelligence.org/team/ - perhaps one of these will contact you with the right person

By the way, do not be surprised if people will hesitate to talk to you. The word "journalist" doesn't exactly bring happy memories in this community. A lot of writing about tech is just clickbait looking for a scandal (and making up something if they don't find one). A famous blogger in our community was harassed by journalists and lost his career over it (1, 2).

Here is someone who isn't shy talking about AI safety on camera: Robert Miles.

If you're really concerned, then just move to california! Its much easier than moving abroad.

I lived in California long enough ago to remember when getting queer-bashed was a reasonable concern for a fair number of people, even in, say, Oakland. It didn't happen daily, but it happened relatively often. If you were in the "out" LGBT community, I think you probably knew somebody who'd been bashed. Politics influence that kind of thing even if it's not legal.

... and in the legal arena, there's a whole lot of pressure building up on that state and local resistance. So far it's mostly money-based pressure, but within a few years, I could easily see a SCOTUS decision that said a state had to, say, extradite somebody accused of "abetting an abortion" in another state.

War in the continental US? No, I agree that's unlikely enough not to worry about.

Civil unrest, followed by violent crackdowns on civil unrest, followed by more violent civil unrest, followed by factional riots, on the other hand...

I'd say I agree with just about all of that, and I'm glad to see it laid out so clearly!

I just also wouldn't be hugely surprised if it turns out something like designing and building remote-controllable self-replicating globally-deployable nanotech (as one example) is in some sense fundamentally "easy" for even an early ASI/modestly superhuman AGI. Say that's the case, and we build a few for the ASI, and then we distribute them across the world, in a matter of weeks. They do what controlled self-replicating nanobots do. Then after a few months the ASI already has an off switch or sleep mode button buried in everyone's brain. My guess is that then none of those hard steps of a war with China come into play. 

To be clear, I don't think this story is likely. But in a broad sense, I am generally of the opinion that most people greatly overestimate how much new data we need to answer new questions or create (some kinds of) new things, and underestimate what can be done with clever use of existing data, even among humans, let alone as we approach the limits of cleverness. 

I think that what you describe as being 2 to 15 percent probable sounds more extreme than what the original post described as being 5 percent probable. You can have "significant erosion" of some groups' rights without leaving the country being the only reasonable option, especially if you're not in those groups. It depends on what you're trying to achieve by leaving, I guess.

Although if I were a trans person in the US right now, especially on medication, I'd be making, if not necessarily immediately executing, some detailed escape plans that could be executed on short notice.

Presumably the "someone dies" means like, within a few years, and not because of x-risk or a major pandemic.

Did you figure out where it's stupid?

I mostly want to point out that many disempowerment/dystopia failure scenarios don't require a step-change from AI, just an acceleration of current trends. 

Do you think that the world is getting worse each year? 

My rough take is that humans, especially rich humans, are generally more and more successful. 

I'm sure there are ways for current trends to lead to catastrophe - line some trends dramatically increasing and others decreasing, but that seems like it would require a lengthy and precise argument. 

Sorry, I can elaborate better on the situation. The big tech companies know that they can pay way more than smaller competitors, so they do. But then that group of megacorp tech (Google, Amazon, Meta, etc.) collude with each other to prevent runaway race dynamics. This is how they're able to optimize their costs with the constraint of salaries being high enough to stifle competition. Here, I was just offering evidence for my claim that big tech is a monopsonistic cartel in the SWE labor market, it isn't really evidence one way or another for the claims I make in the original post.

Oh. I somehow missed/forgot that.

I guess it makes more sense this way. Like, the more transgenders there are in the community, the smaller the fraction of Zizians among them. With the numbers I originally assumed, Ziz's conversion ratio would be shockingly high. Now it makes more sense.

Thank you, this changes my perspective on the situation.

I see where you're coming from, but I think any term in anything anyone writes about alignment can be picked apart ad infinitum. This can be useful to an extent, but beyond a certain point talking about meanings and definitions becomes implementation-specific. Alignment is an engineering problem first and a philosophical problem second.

For example, if RLHF is used to achieve alignment, the meaning of "story" will get solidified through thousands of examples and interactions. The AI will get reinforced to not care about cells or individuals, care about ecosystems and civilizations, and not care as much about the story-of-the-universe-as-a-whole.

If a different alignment method is used, the meaning of "story" will be conveyed differently. If the overall idea is good and doesn't have any obvious failure modes other than simple definitions (e.g. "story" seems to be orders of magnitude simpler to define than "human happiness" or "free will"), I'd consider that a huge success and a candidate for the community to focus real alignment implementation efforts on.