LessWrong 2.0 Reader

You're right. Space is big.

The CSIS wargamed a 2026 Chinese invasion of Taiwan, and found outcomes ranging from mixed to unfavorable for China (CSIS report). If you trust both them and Metaculus, then you ought to update downwards on your estimate of the PRC's strategic ability. Personally, I think Metaculus overestimates the likelihood of an invasion, and is about right about blockades.

Makes sense! Thanks! In that case, we can potentially reduce the width, which might (along with a smaller dataset) help scale saes to understanding mechanisms in big models? 

Unless we build more land (either in the ocean or in space)?

An argument against may be that for some people there's probably a risk of getting addicted / losing control. I'm not familiar with to what degree it's possible to predict such tendencies in advance, but for some people that risk may well outweigh any benefits of arbitrate opportunities or improvements to their calibration.

Come to think of it, I don't think most compute-based AI timelines models (e.g. EPOCH's) incorporate geopolitical factors such as a possible Taiwan crisis. I'm not even sure whether they should. So keep this in mind while consuming timelines forecasts I guess?

I'd rather say that RLHF+'ed chatbots are upon-reflection-not-so-shockingly sycophantic, since they have been trained to satisfy their conversational partner.

Assuming private property as currently legally defined is respected in a transition to a good post-TAI world, I think land (especially in areas with good post-TAI industrial potential) is a pretty good investment. It's the only thing that will keep on being just as scarce. You do have to assume the risk of our future AI(-enabled?) (overlords?) being Georgists, though.

The post is targeted towards the subset of the EA/LW community that is concerned about AI extinction

Ultimately, I think I had a misunderstanding of the audience that would end up reading my post, and I'm still largely ignorant of the psychological nuances of the average LW reader.

Like you implied, I did have a narrow audience in mind, and I assumed that LW's algorithim would function more like popular social media algorithms and only show the post to the subset of the population I was aiming to speak to. I also made the assumption that implications of my post would create the motivation for readers to gain further context through reading links/footnotes for information gaps between me and readers, which seems to be wrong.

For my first sentence where I make the assumption that we subconciously optimize for 

I make a few assumptions.

1. The readers definition of subconscious is the same as my definition. Which is a large leap considering my definition was created through a lot of introspection & self-examination through personal experiences.
2. I assume that if we have the same definition of "subconcious", my claim is self-explanatory. 

Ex 1. When a cancer patient is told they have 5 years left to live, they are heartbroken and their entire lives change.

Ex 2. When an AI researcher predicts 5 years left until the advent of AGI/ASI and the subsequent extinction of the human race, no one takes this seriously.

CstineSublime, if I were to redo this post, do you think I should explain this claim using an example like this instead of making the assumption that readers will automatically make this connection in their mind?


P.S the  footnotes were meant to be definitions similar to what you see when you hover over a complex word on Wikipedia. Let me know if that wasn't clear.

If you could do whole-brain emulation for dolphins, you should be able to generate enough data for unsupervised learning that way.