LessWrong 2.0 Reader

Wiktionary entry

I didn't come up with the title for the essay, but I re-titled this LW post, so thank you for your suggestion. In hindsight, I'll agree that my comment came off as condescending to some extent, so I edited that as well. Regardless of the essay's title, the essay's contents raise serious questions about whether academia is intellectually honest.

I've thought about expanding my sequel essay even further to more precisely quantify and evaluate the research in each academic field, but I ended up not doing this since it would probably take me a week or longer to further detail everything. Another problem was that even if I finished it, people could always say that I failed to evaluate this or that, since there are tens of thousands of papers out there. Another issue is that not everybody agrees on what counts as "fake", as I mentioned in the sequel essay. So even if someone quantified all academic research as best as they can, it's not possible for them to make an overall assessment that a majority of people would agree with.

For these reasons, I don't think it's productive to quantify whether most academic research is true or false or high-quality or low-quality, which would explain why the author didn't do so. I think it's more productive to analyze how academia and the academic research process work and what kind of output such a system is likely to produce. From everything that I've seen across a multitude of fields, my overall impression is that most academic research tends to be low-quality. Blithering Genius's analysis and my own analysis both conclude that that's probably the case for most academic research.

Yes, this seems right to me. The OP says

The key point I will make is that, from a game-theoretic point of view, this race is not an arms race but a suicide race. In an arms race, the winner ends up better off than the loser, whereas in a suicide race, both parties lose massively if either one crosses the finish line.

But from a game-theoretic perspective, it can still make sense for the US to aggressively pursue AGI, even if one believes there's a substantial risk of an AGI takeover in the case of a race, especially if the US acts in its own self interest. Even with this simple model, the optimal strategy would depend on how likely AGI takeover is, how bad China getting controllable AGI first would be from the point of view of the US, and how likely China is to also not race if the US does not race. In particular, if the US is highly confident that China will aggressively pursue AGI even if the US chooses to not race, then the optimal strategy for the US could be to race even if AGI takeover is highly likely.

So really I think some key cruxes here are:

• How likely is AGI (or its descendants) to take over?
• How likely is China to aggressively pursue AGI if the US chooses not to race?

And vice versa for China. But the OP doesn't really make any headway on those.

Additionally, I think there are a bunch of complicating details that also end up mattering, for example:

• To what extent can two rival countries cooperate while simultaneously competing? The US and the Soviets did cooperate on multiple occasions, while engaged in intense geopolitic competition. That could matter if one thinks racing is bad because it makes cooperation harder (as opposed to being bad because it brings AGI faster).
• How (if at all) does the magnitude of the leader's lead over the follower change the probability of AGI takeover (i.e., does the leader need "room to manoeuvre" to develop AGI safely)?
• Is the likelihood of AGI takeover lower when AGI is developed in some given country than in some other given country (all else equal)?
• Is some sort of coordination more likely in worlds where there's a larger gap between racing nations (e.g., because the leader has more leverage over the follower, or because a close follower is less willing to accept a deal)?
• And adding to that, obviously constructs like "the US" and "China" are simplifications too, and the details around who actually makes and influences decisions could end up mattering a lot

It seems to me all these things could matter when determining the optimal US strategy, but I don't see them addressed in the OP.

Based on the words from this post alone -

I think that would depends on what the situation is; in the scenario of price increases, if the business is a monopoly or have very high market power, and the increase is significant (and may even potentially cause harm), then anger would make sense. 

Why is it cheaper for individuals to install some amount of cheap solar power for themselves than for the grid to install it and then deliver it to them, with economies of scale in the construction and maintenance? Transmission cost?

Maybe a better framing would be the economic perspective from Hanson's growth paper: "is AI a complement or is it a substitute?" Does AI assist a human worker (or a human organization), making them more productive, functioning as simply a kind of tool (or 'capital') which multiplies their labor; or does it replace that human worker/organization? When it's the former, it may indeed take a very long time; but the latter can happen instantly.

No one can force a freelance artist to learn to use Photoshop or how to best use some snazzy new feature, and artists will be learning the ins-and-outs of their new technologies and workflows for many decades to come and slowly becoming more productive thanks to their complementing by digital illustration tools. Whereas on the other hand, their employers can replace them potentially in minutes after the next big Midjourney upgrade.*

More historically, in colonization, a group of settlers may simply arrive literally overnight in their wagons and set up a new town (eg. a gold rush boomtown), and begin replacing the local indigenous peoples, without any sort of centuries-long gradual '+2% local per capita GDP growth per year until convergence' using only the original local indigenous people's descendants.

* A personal example: when I wanted more fancy dropcaps for Gwern.net, I was contacting human artists and trying to figure out how much it would cost and what the workflow was, and how many thousands of dollars & months of back-and-forth a good dropcap set might cost, and if I would have to settle for instead something like 1 custom dropcap per essay. When Midjourney became reasonably adequate at v5 & DALL-E at 3, I didn't spend several years working with artists to integrate AI into their workflow and complement their labor... I stopped my attempt to use them that night, and never looked back. At this point, I'm not sure how many artists or font designers I would want to use even if they were free, because it means I don't have to deal with folks like Dave or have one of my projects delayed or killed by artists, or the hassle of all the paperwork and payments, and I get other benefits like extremely rapid iteration & exploration etc.

Awesome, thank you! I'm not sure if we're going to correct this; it's a pain in the butt to fix, especially in the YouTube version, and Elizabeth (who has been doing all the editing herself) is sick right now.

who

redundant "who"s in bullets

I think normally "agile" would fulfill the same function (per its etymology), but it's very entangled with agile software engineering.

Very nice piece, and thank you for your service

"cloudy brother of bacteria" should probably be "cloudy broth of bacteria".

Do you understand mathematically what operation we're doing when we say two species or organisms have xx% similar genomes? Each genome is, I guess, several sequences of ATCG, but how do you get a percent similarity for two sequences of different lengths?