LessWrong 2.0 Reader

TLDR:
Here's all the ways in which you're right, and thanks for pointing these things out!

At a meta-level, I'm *really* excited by just how much I didn't see your criticism coming. I thought I was thinking carefully, and that iterating on my post with Claude (though it didn't write a single word of it!) was taking out the obvious mistakes, but I missed so much. I have to rethink a lot about my process of writing this.

I strongly agree that I need a *way* more detailed model of what "memetic evolution" looks like, when it's good vs bad, and why, whether there's a better way of phrasing and viewing it, dig into historical examples, etc.

I'm curious if social media is actually bad beyond the surface -- but again I should've anticipated "social media kinda seems bad in a lot of ways" being such an obvious problem in my thinking, and attended to it.

Reading it back, it totally reads as an argument for "more information more Gooder", which I didn't see at all. (generally viewing the post as "more X is always more good" is also cool as in, a categorization trick that brings clarity)

I think a good way to summarize my mistake is that I didn't "go all the way" in my (pretty scattered) lines of thinking.

 You're on your way to thinking critically about morality, coordination and epistemology, which is great!

Thanks :)  A big part of why I got into writing ideas explicitly and in big posts (vs off-hand Tweets/personal notes), is because you've talked about this being a coordination mechanism on Discord.

OK, nice.

I didn't think through carefully what I mean by 'density' other than to say: I mean '# of chokepoints the defender needs to defend, in a typical stretch of frontline' So, number of edges in the network (per sq km) sounds like a reasonable proxy for what I mean by density at least.

I also have zero hours of combat experience haha. I agree this is untested conjecture & that reality is likely to contain unexpected-by-me surprises that will make my toy model inaccurate or at least incomplete.



 

(For reference I think Tsvi and GeneSmith have much more relevant knowledge for evaluating the chance of superbabies being feasible and I updated my guess to like 78%.)

(As it happens I also became more optimistic about the orca plan (especially in terms of how much it would cost and how long it would take, but also a bit in how likely I think it is that orcas would actually study science) (see footnote 4 in post). For <=30y timelines I think the orca plan is a bit more promising, though overall the superbabies plan is more promising/important. I'm now seriously considering pivoting to the orca plan though.) (EDIT: tbc I'm considering pivoting from alignment research, not superbaby research.)

Thanks for the feedback! Have editted the post to include your remarks.

It gets very complicated when you add in incentives and recognize that science and scientists are also businesses.  There's a LOT of the world that scientists haven't (or haven't in the last century or so) really tried to prove, replicate, and come to consensus on.

EDIT: Due to the incoming administration's ties to tech investors, I no longer think an AI crash is so likely. Several signs [EA(p) · GW(p)] IMHO point to "they're gonna go all-in on racing for AI, regardless of how 'needed [EA · GW]' it actually is".

Yes for the first half, no for the second.  I would reply 1/2, but not JUST because of conventional probability theory.  It's also because the unstated parts of "what will resolve the prediction", in my estimation and modeling, match the setup of conventional probability theory.  It's generally assumed there's no double-counting or other experience-affecting tomfoolery.

Could it be a good idea to enable a file uploading feature to LessWrong? (eg a file uploading feature of PDFs or certain images/media]. Could help against link rot, for example (and make posts from long ago last longer - I say this as someone who edits old posts to make them timeless).

The customer doesn't pay the fee directly. The vendor pays the fee (and passes the cost to the customer via price). Sometimes vendors offer a cash discount because of this fee.

Don't overthink it. Two downvotes (or maybe one strong downvote) just means that there were one or two people who didn't like the answer, and the rest either didn't notice it or didn't care enough to vote.

I understand that it sucks, but in general, if few people vote on a thing, there is a lot of noise.