Contrarianism and reference class forecasting

Sometimes "looking at the thing itself" is too costly or too difficult. How can the proverbial "bright sixteen-year-old" sitting in a high school classroom figure out the truth about, say, the number of protons in an atom of gold, without having to accept the authority of his textbooks and instructors? If there were a bunch of well-funded nutcases dedicated to arguing that gold atoms have seventy-eight protons instead of seventy-nine, the only way you can really judge who's correct is to judge the relative credibility of the people presenting the evidence. After all, one side's evidence could be completely fraudulent and you'd have no way of knowing that.

Far too often, reference classes and meta-level discussions are all we have.

How should we unpack black boxes we don't have yet? For example a non-neural language capable self-maintaining goal-oriented system*.

We have a surfeit of potential systems (with different capabilities of self-inspection and self-modification) with no way to test whether they will fall into the above category or how big the category actually is.

*I'm trying to unpack AGI here somewhat

I estimate that even fairly bad reference class / outside view analysis is still far more reliable than the best inside view that can be realistically expected. People are just spectacularly bad at inside view analysis, and reference class analysis puts hard boundaries within which truth is almost always found.

I think you're completely right, this is a special case of the problem of induction. The Stanford Encyclopedia of Philosophy has a wonderfully exhaustive article about it that also discusses subjective Bayesianism at length. Among other things, that article offers a simple recommendation for taw's original problem: intersect your proposed reference classes to get a smaller and more relevant reference class.

Agreed with the first part and with the heuristic, but taw is using the possibility of politicization as an element of reference class membership. Honestly, I wouldn't even consider global warming to be a "political issue". The science seems completely trivial to understand at the object level.

Not really, induction problems philosophers talk about are pure theory, and totally irrelevant to daily life. Everybody knows blue/green are correct categories, while grue/bleen are not.

Figuring out proper reference class on the other hand, is a serious problem of applied rationality.

Other example: doomseday-ee predictions. Overpopulation, peak oil, overfishing: in each case, the science in the prediction was pretty right.

This is not what they were about. What they predicted was massive suffering in each case. Overpopulation doomsdayers predicted food and resource shortages, wars for land and water and such; peak oilers predicted total collapse of economy, death of over half of humanity, and such. Other than for their supposedly massive consequences peak oil is as interesting as peak typewriters, that is not at all unless you work in oil/typewriter industry.

By the way false predictions of underlying process were false in all three cases you mention - population growth is sublinear for quite some time, peak oil reliably doesn't take place on any of predicted dates, and total fish production is increasing via aquaculture - or true in only the most restricted way, far more restricted than what was claimed - population did increase at all, old oil fields are depleting, wild fish production is not increasing - but this is irrelevant - the core of doomsdayer predictions is the doom part, which almost invariably doesn't happen.

I would estimate that vast majority of carnivorous predators are tiny insects and such, so the class is even less dangerous than land mammals class. ;-) On the other hand class of "animals bigger than me" tends to be quite dangerous.

As far as I understand history of this claim, back during the Cold War it was common to predict that even small a scale nuclear exchange will get the world back to long term Ice Age, due to widespread urban fires. Similar predictions were even made about Kuwait oil wells fires in 1991 (it's a good model, as the effect was not supposed to be related to nuclear explosions as such, just resulting fires).

It turns out from more recent models, and actual data from the Gulf War that the actual magnitude of cooling is orders of magnitude smaller than what was predicted, and there was never any genuine research that really suggested levels that were widely claimed; the most straightforward explanation is that people opposed to nuclear weapons wanted to exaggerate its effect to scare people off. It might have been a calculated lie, or something they genuinely wanted to believe - the point is that politicized science is not very accurate even if you agree with its political goals.

"things will stay just the same as they are today" would be a horrible horrible predictor for the last 40 years.

Check gapminder 1969 to 2009 for the poorest end how drastic and cornucopian the changes were for most of them. For the rich end, Internet, mobiles, and other wonderful technology counts very much as cornucopia.

That's not the Doomsday I was talking about, just prediction of massive suffering due to one cause or another, be in overpopulation, nuclear war, biological war, food shortages, water shortages, oil shortages, phosphorus shortages, guano shortages, whale oil shortages, rare earth metal shortages, shortages of virtually every commodity, flu pandemic, AIDS pandemic, mass scale terrorism, workers' revolutions, Barbarian takeover, Catholic takeover, Communist takeover, Islamic takeover, or whatnot, just to name a few.

Pretty much none of them caused the massive suffering and collapse of civilization predicted. Most do not involve end of humanity, so invocations of the antropic principle are misguided.

"Just because energy (mainly coal and oil) companies don't like the policy implications of climate science and are willing to pay lots of people to speak ill of it, shouldn't make it a politicized science."

It seems as though energy companies have an incentive to downplay science that provides justification for limiting CO2, but don't scientists with government funding have incentive to play up science that provides justification for an increase in government power? How could we find out the magnitude of there effects without actually understanding the research ourselves?

You just confirm my point. The very fact that you use phrases like "policy implications of climate science", and "subject's ideological enemies" shows it's a highly politicized field. You wouldn't say "policy implications of quantum physics" or "chemistry's ideological enemies".

In case you didn't follow Climategate, it look that scientists from East Anglia University engaged in politics a lot, including dirty politics; and they were nothing like neutral scientists merely seeking the truth and letting others deal with policy. You may find their actions warranted due to some greater good, or not, but it's not normal scientific practice, and I'd be willing to bet against pretty high rates that you would not find anything like that on any evolutionary biology department.

That doesn't mean their findings are wrong. There are plenty of highly politicized issues where the mainstream is right or mostly right, but this rate is significantly lower than for non-politicized science. For example mainstream accounts of histories of most nations tend to be strongly whitewashed, as it's politically convenient. They are mostly accurate when it comes to events, but you can be fairly sure there are some systemic distortions. That's the reference class in which I put climate science - most likely right on main points, most likely with significant distortions, and with non-negligible chance of being entirely wrong.

On the other hand the moment climate scientists switch from talking about climate to talking about policy or impact of climate change of human well-being, I estimate that they're almost certainly wrong. There is no reference class I can think of which suggests otherwise, and the closest reference class of Doomsday predictors has just this kind of track record.

If you want some more, I did blog a bit about climate change recently: 1, 2, 3.

I replied to your point about evolutionary biology here.

Would that not bias the results? Category like "mainstream science" bundles together chemistry with virtually impeccable record, and psychology with highly dubious one. Using category like that we'll greatly underestimate certainty of chemical predictions, and greatly overestimate certainty of psychological ones.

What I wanted to say is that we move from supporters and opponents arguing about particulars of a situation to supporters and opponents arguing about proper reference class. Which might be an improvement, but it doesn't solve the issue.

People who agree with a political bias generally don't believe there's a bias. Witness all the academic "liberals" who accept global warming, CFC fearmongering, overpopulation, and resource exhaustion without question and attack people who question any of these received wisdoms.

What are you basing this claim of uncontroversial status on?

http://en.wikipedia.org/wiki/Snyderman_and_Rothman_(study)

Surveys of psychometricians and other relevant psychologists have never shown a consensus in support of Jensen's views on group differences. At most, a more moderate version of his position (some genetic component, perhaps small) has held plurality or bare majority support in the past (but might not anymore, in light of work such as Flynn's) while remaining controversial.