The Median is Less than the Average

post by lsusr · 2021-02-15T02:28:39.788Z · LW · GW · 16 comments

In The Economics of Media [LW · GW] I discussed how news is written by large organizations including businesses, governments and major political parties.

The same was true in the age of paper media except with the addition of more powerful local monopolies. Much of the United States (and likely the rest of the world) had one (or few) major newspaper(s). Chaos favors the underdog. When a single company controls an industry, it is incentivized to promote stability. It is more important for a monopoly newspaper to censor offensive news than to create interesting content.

We often think of "censorship" as suppression of politically sensitive topics. Actually, most censorship pertains to merely offensive topics. In the forewords and afterwords to Gary Larson's The Far Side collections you'll find incident after incident of his work (which is far less titillating than xkcd) getting suppressed by puritan surburban readers—usually for reasons unrelated to politics. Paper newspapers maintained their monopolies by regurgitating bland moderation.

Centralized anything is a single point of failure. Is is easy to suppress a centralized operation like Bing, Facebook, YouTube, WeChat, a school or The Seattle Times. It is impossible to censor the entire Internet. Before the Internet, the only opinions you could conveniently access were those which passed all the filters built into the centralized paper publishers. Now you can access all the opinions.

Increased information makes smart people smarter and stupid people stupider. If you are smart then increased access to information is unequivocably a good thing. You can master basically any technical subject just from the books available on two websites. If you are stupid then increased information makes the world a confusing place because it's harder to just conform to the party line. Increased information means there is more media specifically designed to dupe stupid people. The Internet increases the variance of human intelligence.

Human intelligence is asymmetrically distributed. Intelligence is capped on the bottom but it is long-tailed at the top. It is more like a Poisson distribution than a Gaussian distribution. The median is less than the average. As variance increases, the difference gap between average and median widens. Increased variance in intelligence is a pressure behind increasing income inequality. It makes the world less equal and more equitable.


Comments sorted by top scores.

comment by Donald Hobson (donald-hobson) · 2021-02-15T09:57:17.107Z · LW(p) · GW(p)

Intelligence is capped on the bottom but it is long-tailed at the top.

This seems wrong to me. From an evolutionary perspective, there shouldn't be complex new structures that are only occasionally present. 

There is a modal human brain. Most particular brains have a few small errors that make them slightly worse. Some have a small mutation that makes them slightly better. The long tail stretches downwards, into the people with severe learning disabilities. 

Replies from: donald-hobson, Daniel V, Ericf
comment by Donald Hobson (donald-hobson) · 2021-02-15T10:19:44.125Z · LW(p) · GW(p)

However, It depends what you are measuring, and what scale you are using. Use a scale like Busy Beaver IQ and almost everyone is below average. Because you are streaching out differences at the top of the scale.

There is also a difference between biological intelligence and optimization power. A caveman with the genes for being unusually smart still won't know much science. It is at least plausible that the median is less than the mean here, but again, it depends on the scale used.

comment by Daniel V · 2021-02-15T16:20:44.399Z · LW(p) · GW(p)

OP's claim is that intelligence is positively skewed. Counter-points are "most brains are slightly worse"  (Donald Hobson) and "you oversample the high-intelligence people, so your claim is biased because of availability" (Ericf).

Both of these counter-points agree with, rather than disagree with, lsusr's point. Most brains are slightly worse implies positive skew and to the extent that lsusr oversamples high-intelligence people, they are underestimating how positively skewed intelligence is yet still conclude it is positively skewed (caveat: as Donald Hobson says, the measurement approach can be really important here, but for the sake of argument let's say lsusr is talking about latent intelligence, and our measures just need to catch up with the theory).

Ericf also makes another interesting point- "variation in low intelligence is less identifiable than variation in high intelligence," 160 vs. 130 IQ people will act differently, but 40 vs. 70 IQ people won't so much, or at least the IQ test is better at delineating on the high end than low end. I am no expert on the measurement of intelligence, but this point probably shouldn't just be taken at face value- for example, individuals with Down's syndrome consistently have IQs less than 70 and getting below 70 is rare, as expected since IQ is designed to be Gaussian. But the implication of that is that as rare (and therefore difficult to dig into) as low IQs are, high IQs are...equally rare (and therefore difficult to dig into).

I agree that OP's claim should also be subjected to scrutiny -simply saying intelligence is positively skewed doesn't make it so- but I also don't find the present set of counter-points either that contradictory or that convincing either. Just my two cents.


comment by Ericf · 2021-02-15T13:24:12.932Z · LW(p) · GW(p)

I think you are right, and OP fell victim to the Availability heuristic. 4 SD above average is 160 IQ, which is a normal level of "very smart." Everyone reading this probably personally knows someone around that level. And we can almost see the impact a star programmer, or an Einstein, makes on the world.

4 SD below average is a) not well measured because "nobody" cares once you go below 70 (2 SD); b) not going to be making an impact on society; and c) not going to present too differently than an IQ 70 person - you don't know if it took the person stocking shelves 1 hour, 1 day, or 1 year of instruction to reach maximum shelf-stocking skill.

comment by crl826 · 2021-02-15T03:50:59.540Z · LW(p) · GW(p)

Now you can access all the opinions.


This is becoming less and less true. Removing people from social media. Removing entire websites. Even denying people the ability to participate in financial systems.

Replies from: Ericf
comment by Ericf · 2021-02-15T04:18:23.555Z · LW(p) · GW(p)

As the OP wrote: Is is easy to suppress a centralized operation like Bing, Facebook, YouTube, WeChat, a school or The Seattle Times. It is impossible to censor the entire Internet.

Replies from: crl826
comment by crl826 · 2021-02-15T04:42:01.785Z · LW(p) · GW(p)

As I wrote: that is becoming less and less true. 

I'm surprised that that was taken so negatively. I'm not exactly sure why.

Replies from: benmann
comment by benmann · 2021-02-15T04:54:29.272Z · LW(p) · GW(p)

Because the article already rebuts your argument. Before the internet, fringe voices would not have anywhere to go. You are right that some fringe voices get removed from a few big platforms, but you can always start a blog or go to a smaller platform.

Replies from: crl826
comment by crl826 · 2021-02-15T05:14:40.305Z · LW(p) · GW(p)

Except blogs have been removed from the internet. And entire, smaller platforms are wiped from the internet.

It's becoming less true that you can always move to a blog or a smaller platform. That's what I said. Seems true and not in the article.

But I suppose I will live to fight another day.

Replies from: cistran, Strawperson
comment by cistran · 2021-02-15T17:17:32.821Z · LW(p) · GW(p)

And yet, Piratebay stays online.

Replies from: crl826
comment by crl826 · 2021-02-20T04:57:15.190Z · LW(p) · GW(p)

This is a good, fair point (unlike the person who wrote me and told me I was spreading Nazi propaganda). Thank you.

I confess I am not tech savvy enough to validate these arguments, but I have heard that Piratebay is much simpler (text files that point to other files) and is much easier to keep play cat and mouse games than, say, a social network.

But, either way, you are right to point out that a determined opponent can keep up a fight for awhile.

comment by Strawperson · 2021-02-17T13:31:45.155Z · LW(p) · GW(p)

I think both claims are true but on different time scales: (1) yes, the information and discourse readily available to the median internet user is less free and diverse today than it was a decade or two ago, but also, (2) this information and discourse is still more free and diverse than what the vast majority of people would encounter anywhere in the media pre-internet.

There are discussions to be had about which of these trends are more important, for "society in general", or in more specific context, or how things will play out in the future, but I find it very hard to believe that these facts are not both true.

Replies from: crl826
comment by crl826 · 2021-02-20T04:57:54.061Z · LW(p) · GW(p)

This makes a lot of sense. Thanks.

comment by noggin-scratcher · 2021-02-15T10:31:53.575Z · LW(p) · GW(p)

In the forwards and afterwards

(forewords and afterwords)*

Replies from: lsusr
comment by lsusr · 2021-02-15T10:36:41.327Z · LW(p) · GW(p)

Fixed. Thanks.

comment by seed · 2021-02-23T18:53:27.969Z · LW(p) · GW(p)

How do you measure intelligence though? Obviously you don't mean IQ, since IQ test scores are deliberately calibrated to be normally distributed.

Generally, I feel like all these bold claims need some supporting evidence. E.g.

Increased information makes smart people smarter and stupid people stupider.

Citation needed?

comment by cousin_it · 2021-02-16T12:11:20.529Z · LW(p) · GW(p)

Increased information makes smart people smarter and stupid people stupider.

I can't think of any kind of information that has this effect. Technical information makes people smarter, all people. Entertainment, politics, and politics-entertainment (which together comprise most of the internet) make people stupider. The harm of entertainment is that you spend lots of time receiving stimuli without putting in any action or thought, and the harm of politics-entertainment is that you also end up resenting other people. And behind it all is the harm of advertising, which makes you want stuff you don't need and envy other people for having it. That's a lot of harm.