LessWrong 2.0 Reader

My rough first attempt at explaining the apparent paradox of poverty continuing to exist despite a ~100x increase in productivity:

Humanity as a whole may have gotten 100x more productive, but most people aren't able to individually contribute 100x more value to the economy (at least not in a way that they are monetarily compensated for).  It seems like the increase in productivity largely takes the form of finding more efficient ways to coordinate the efforts of many people (e.g. factories) rather than individuals being able to produce more value on their own, and that the few people doing the coordinating are reaping the benefits of this much more than the many people being coordinated.  Even in areas where the potential for individual productivity has greatly increased (e.g. digitally producing games, videos, music, etc.), attention remains scarce, people largely pay attention to things that already have attention, and again only a small number of people are able to produce such content and have a large number of people consume and pay for it.

If humanity's 100x productivity increase took the form of 100% of people becoming 100x more productive, and poverty persisted, that would indeed be pretty strong evidence for some sort of Poverty Restoring Force, but my sense is that it's a closer approximation of what actually happened to say that 1% of people became 10,000x more productive, in which case observing that many of the other 99% who were previously in poverty continue to be in poverty is not surprising - it's not an observation that requires one to postulate Poverty Restoring Forces in order to explain.  For this reason, I continue to lean towards optimism about UBI being an effective means of reducing poverty.

You say "higher numbers for polyamorous relationships" which is contrary to "If you're polyamorous, but happen to have one partner, you would also put 1 for this question."

However, I find myself appealing to basic logical principles like the law of non-contradiction.

The law of non contradiction isn't true in all "universes" , either. It's not true in paraconsistent logic, specifically.

I agree that inventing new arguments for X that sound kind-of plausible to you on the surface level, and which you imagine would work well on a given demographic, is not a recipe for good communication. Such arguments are "artificial", they're not native citizens of someone's internally consistent world-model, and it's going to show and lead to unconvincing messages that fall apart under minimal scrutiny.

That's not what I'm arguing for. The case for the AGI risk is overdetermined: there are enough true arguments for it that you can remove a subset of them and still end up with an internally consistent world-model in which the AGI risk is real. Arguably, there's even a set of correct arguments that convinces a Creationist, without making them not-a-Creationist in the process.

Convincing messaging towards Creationists involves instantiating a world-model in which only the subset of arguments Creationists would believe exist, and then (earnestly) arguing from within that world-model.

Edit: Like, here's a sanity-check: suppose you must convince a specific Creationist that the AGI Risk is real. Do you need to argue them out of Creationism in order to do so?

Yeah, this would either need options for many countries or one schema for many countries.

Asking whether they voted or not in a national election is straight forward enough, and there's been past questions like that.

"Voting Did you vote in your country's last major national election?"

Hrm. I parse this as part of an example: if you are partnered and monogamous (and faithful!) then you should put down 1. If you're polyamorous, but happen to have one partner, you would also put 1 for this question. There's a Relationship Styles question that gets at what people prefer.

Do you think this example will confuse people?

I don't agree with the following claims (which might misrepresent you):

• "Skill levels" are domain agnostic.
• Frontier oversight, control, evals, and non-"science of DL" interp research is strictly easier in practice than frontier agent foundations and "science of DL" interp research.
• The main reason there is more funding/interest in the former category than the latter is due to skill issues, rather than worldview differences and clarity of scope.
• MATS has mid researchers relative to other programs.

No, I think this kind of very naive calculation does predictably result in worse arguments propagating, people rightfully dismissing those bad arguments (because they are not entangled with the real reasons why any of the people who have thought about the problem have formed beliefs on an issue themselves), and then ultimately the comms problem getting much harder.

I am in favor of people thinking hard about these issues, but I think exactly this kind of naive argument are in an uncanny valley where I think your comms gets substantially worse.

Perhaps. Admittedly, I don't have a solid model of whether a median American claiming to be a Creationist in surveys would instantly dismiss a message if it starts making arguments from evolution.

Still, I think the general point applies:

• A convincing case for the AGI Omnicide Risk doesn't have to include arguments from human evolution.
• Arguments from human evolution may trigger some people to instinctively dismiss the entire message.
• If the fraction of such people is large enough, it makes sense to have public AI-Risk messages that avoid evolution-based arguments when making their case.

I don't think it makes sense to compare Google intern salary with AIS program stipends this way, as AIS programs are nonprofits (with associated salary cut) and generally trying to select against people motivated principally by money. It seems like good mechanism design to pay less than tech internships, even if the technical bar for is higher, given that value alignment is best selected by looking for "costly signals" like salary sacrifice.

I don't think the correlation for competence among AIS programs is as you describe.