LessWrong 2.0 Reader

Suppose people's probability of solving a task is uniformly distributed between 0 and 1. That's a thin-tailed distribution.

Now consider their probability of correctly solving 2 tasks in a row. That will have a sort of triangular distribution, which has more positive skewness.

If you consider e.g. their probability of correctly solving 10 tasks in a row, then the bottom 93.3% of people will all have less than 50%, whereas e.g. the 99th percentile will have 90% chance of succeeding.

Conjunction is one of the two fundamental ways that tasks can combine, and it tends to make the tasks harder and rapidly make the upper tail do better than the lower tail, leading to an approximately-exponential element. Another fundamental way that tasks can combine is disjunction, which leads to an exponential in the opposite direction.

When you combine conjunctions and disjunctions, you get an approximately sigmoidal relationship. The location/x-axis-translation of this sigmoid depends on the task's difficulty. And in practice, the "easy" side of this sigmoid can be automated or done quickly or similar, so really what matters is the "hard" side, and the hard side of a sigmoid is approximately exponential.

People from “primitive” societies give more than people from more developed societies, and the more primitive the society, the stronger the effect.

I am not sure what you intended to say here, but the word "primitive" definitely looks like a red flag. As I don't think I am the only one to believe this, I would ask you to please change the wording or delete this sentence.

That's what I mean (I'm talking about the input/output behavior of individual neurons).

Ah, I see. Nvm then.

Yeah I know that's why I said that if a major effect was through few significantly deleterious mutations this would be more plausible. But i feel like human intelligence is even more heavitailed than what one would predict given this hypothesis.

If you have many mutations that matter, then via central limit theorem the overall distribution will be roughly gaussian even though the individual ones are exponential.

(If I made a mistake maybe crunch the numbers to show me?)

(initially misunderstood what you mean where i thought complete nonsense.)

I don't understand what you're trying to say. Can you maybe rephrase again in more detail?

The British weren't much more compassionate. The native populations of North America and Australia were almost completely destroyed and replaced with Europeans. Under British rule in India, tens of millions died from many famines, which instantly stopped after independence.

Colonialism didn't end due to benevolence. Wars for colonial liberation continued well after WWII and were very brutal, the Algerian war for example. I think the actual reason is that colonies stopped making economic sense.

So I guess the difference between your view and mine is that I think colonialism kept going (and being basically the worst thing that happened in history) as long as it benefited the dominant group overall. Benevolence or malevolence didn't come into it much. And if we get back to the AI conversation, my view is that when AIs become more powerful than people and can use resources more efficiently, the systemic gradient in favor of taking everything away from people will be just way too strong. It's a force acting above the level of individuals (hmm, individual AIs) - it will affect which AIs get created and which ones succeed.

What I think is more likely than EA pivoting is a handful of people launch a lifeboat and recreate a high integrity version of EA.

Thoughts on how this might be done:

• Interview a bunch of people who became disillusioned. Try to identify common complaints.

• For each common complaint, research organizational psychology, history of high-performing organizations, etc. and brainstorm institutional solutions to address that complaint. By "institutional solutions", I mean approaches which claim to e.g. fix an underlying bad incentive structure, so it won't require continuous heroic effort to address the complaint.

• Combine the most promising solutions into a charter for a new association of some kind. Solicit criticism/red-teaming for the charter.

• Don't try to replace EA all at once. Start small by aiming at a particular problem present in EA, e.g. bad funding incentives, criticism (it sucks too hard to both give and receive it), or bad feedback loops in the area of AI safety. Initially focus on solving that particular problem, but also build in the capability to scale up and address additional problems if things are going well.

• Don't market this as a "replacement for EA". There's no reason to have an adversarial relationship. When describing the new thing, focus on the specific problem which was selected as the initial focus, plus the distinctive features of the charter and the problems they are supposed to solve.

• Think of this as an experiment, where you're aiming to test one or more theses about what charter content will cause organizational outperformance.

I think it would be interesting if someone put together a reading list on high-performing organizations, social movement history, etc. etc. I suspect this is undersupplied on the current margin, compared with observing and theorizing about EA as it exists now. Without any understanding of history, you run the risk of being a "general fighting the last war" -- addressing the problems EA has now, but inadvertently introduce a new set of problems. Seems like the ideal charter would exist in the intersection of "inside view says this will fix EA's current issues" and "outside view says this has worked well historically".

A reading list might be too much work, but there's really no reason not to do an LLM-enabled literature review of some kind, at the very least.

I also think a reading list for leadership could be valuable. One impression of mine is that "EA leaders" aren't reading books about how to lead, research on leadership, or what great leaders did.

Thanks, I mostly agree.

But even in colonialism, individual traits played a role. For example, compare King Leopold II's rule over the Congo Free State vs. other colonial regimes. 

While all colonialism was exploitative, under Leopold's personal rule the Congo saw extraordinarily brutal policies, e.g., his rubber quota system led soldiers to torture and cut off the hands of workers, including children, who failed to meet quotas. Under his rule,1.5-15 million Congolese people died—the total population was only around 15 to 20 million. The brutality was so extreme that it caused public outrage which led other colonial powers to intervene until the Belgian government took control over the Congo Free State from Leopold.

Compare this to, say, British colonial administration during certain periods which, while still overall morally reprehensible, saw much less barbaric policies under some administrators who showed basic compassion for indigenous people. For instance, Governor William Bentinck in India abolished practices like sati (widows burning themselves alive) and implemented other humanitarian reforms. 

One can easily find other examples (e.g. sadistic slave owners vs. more compassionate slave owners). 

In conclusion, I totally agree that power imbalances enabled systemic exploitation regardless of individual temperament. But individual traits significantly affected how much suffering and death that exploitation created in practice.^[1] 

1. ^{^}

   Also, slavery and colonialism were ultimately abolished (in the Western world). My guess is that those who advocated for these reforms were, on average, more compassionate and less malevolent than those who tried to preserve these practices. Of course, the reformers were also heavily influenced by great ideas like the Enlightenment / classic liberalism. 

If each deleterious mutation decreases the success rate of something by an additive constant, but you need lots of sequential successes for intellectual achievements, then intellectual formidability is ~exponentially related to deleterious variants.

(Thanks. I don't think this is necessarily significant evidence against my hypothesis (see my comment on GeneSmith's comment.)

Another confusing relevant piece of evidence I thought I throw in:

Human intelligence seems to me to be very heavytailed. (I assume this is uncontrovertial here, just look at the greatest scientists vs great scientists.)

If variance in intelligence was basically purely explained by mildly-delterious SNPs, this would seem a bit odd to me: If the average person had 1000SNPs, and then (using butt-numbers which might be very off) Einstein (+6.3std) had only 800 and the average theoretical physics professor (+4std) had 850, I wouldn't expect the difference there to be that big.

It's a bit less surprising on the model where most people have a few strongly delterious mutations, and supergeniuses are the lucky ones that have only 1 or 0 of those.

It's IMO even a bit less surprising on my hypothesis where in some cases the different hyperparameters happen to work much better with each other -- where supergeniuses are in some dimensions "more lucky than the base genome" (in a way that's not necessarily easy to pass on to offspring though because the genes are interdependent, which is why the genes didn't yet rise to fixation). But even there I'd still be pretty surprised by the heavytail.

The heavytail of intelligence really confuses me. (Given that it doesn't even come from sub-critical intelligence explosion dynamics.)

Yep, Claude sure is a pretty good coder: Wang Tile Pattern Generator

This took 1 initial write and 5 change requests to produce. The most manual effort I had to do was look at unicode ranges and see which ones had distinctive-looking glyphs in them. (Sorry if any of these aren't in your computer's glyph library.)