Climate science: how it matters for understanding forecasting, materials I've read or plan to read, sources of potential bias

post by VipulNaik · 2014-07-07T16:18:57.061Z · LW · GW · Legacy · 8 comments

Contents

  Why am I looking at climate forecasting?
  The sorts of questions I hope to answer by the end of this inquiry
  Sources
  Courses or full-fledged reviews
  Books about climate change aimed at a popular audience
  Books about specific controversies surrounding climate change
  Book chapters
  IPCC reports
  Blogs and websites
  Papers
  Potential for bias and inaccuracy
  Looking for suggestions
None
8 comments

As part of a review of forecasting, I've been looking at weather and climate forecasting (I wrote one post on weather forecasting and another on the different time horizons for weather and climate forecasting).

Climate forecasting is turning out to be a fairly tricky topic to look into, partly because of the inherent complexity of the task, and partly because of the politicization surrounding Anthropogenic Global Warming (AGW).

Due to the complexity and the potential for bias, I decided to disclose what materials I've read and my potential sources of bias.

Why am I looking at climate forecasting?

Climate forecasting, and the debate surrounding what'll happen to the climate and how human choices today can shape it, is one of the biggest examples of a long-range forecasting effort that has attracted widespread attention, both in terms of the science and the policy and political implications. Understanding how it was done can give insights into the ability of humans to make forecasts about the long-run future (on the decadal or centennial timescale) in the face of considerable uncertainty, and use those forecasts to drive decisions today. This would be relevant for other long-range forecasting problems, such as (possibly) friendly AI. Note though that my focus isn't driven by finding parallels with any other specific forecasting problem, such as friendly AI.

The sorts of questions I hope to answer by the end of this inquiry

The following are questions to which I hope to state relatively clear answers by the end:

The following are questions to which I may not obtain clear answers, but I'll be looking for and reporting information on them because they influence the answers to the preceding questions:

Sources

Courses or full-fledged reviews

Books about climate change aimed at a popular audience

Books about specific controversies surrounding climate change

Book chapters

IPCC reports

Blogs and websites

I reference here only the blogs and websites I've identified as places to check out, rather than ones where I chanced upon an isolated blog post by link-traipsing or searching the web.

Papers

I read many papers from a diverse array of sources. I arrived at most papers either by clicking links on one of the blogs or websites mentioned above, or using Google or Google Scholar searches for specific topics. Any paper that I use as input to my opinion in a specific post will be explicitly linked in that post.

Potential for bias and inaccuracy

Looking for suggestions

As always, I'm happy to hear suggestions. In particular, I am interested in suggestions on these fronts:

8 comments

Comments sorted by top scores.

comment by Stefan_Schubert · 2014-07-07T16:29:00.703Z · LW(p) · GW(p)

More generally, to what extent have people's beliefs about the possible political consequences about specific outcomes affected the science in ways that are not epistemically justified? For instance, do people who are more risk-averse tend to exaggerate the harms, so that they can convince a less risk-averse public to take action?

That seems likely, given what we know of human biases, though I think that people would normally do that unconsciously rather than intentionally . This paper by Alhakami and Slovic shows that people who think, e.g. that nuclear power is risky don't think it's beneficial, while those who think it's beneficial think it's risky (they explain this by reference to the halo effect):

Judgments of risk and judgments of benefit have been found to be inversely related. Activities or technologies that are judged high in risk tend to be judged low in benefit, and vice versa. In the present study, we examine this inverse relationship in detail, using two measures of relationship between risk and benefit. We find that the inverse relationship is robust and indicative of a confounding of risk and benefit in people's minds. This confounding is linked to a person's overall evaluation of an activity or technology. Theoretical and practical implications of this risk-benefit confounding are discussed.

Replies from: VipulNaik, torekp
comment by VipulNaik · 2014-07-07T16:46:39.282Z · LW(p) · GW(p)

Yes, I would expect this to be likely too. But I'm interested in collecting specific object-level evidence of this being the case. (I've already collected some information, but I'm still processing and checking it).

comment by torekp · 2014-07-12T16:04:32.160Z · LW(p) · GW(p)

while those who think it's beneficial think it's [not] risky

-- I think you meant.

comment by rxs · 2014-07-08T11:14:23.060Z · LW(p) · GW(p)

John Baez's Azimuth project/community might be of interest to you.

http://johncarlosbaez.wordpress.com/

comment by drnickbone · 2014-07-07T22:05:38.320Z · LW(p) · GW(p)

I've noticed that you've listed a lot of secondary sources (books, blogs, IPCC summaries) but not primary sources (published papers by scientists in peer-reviewed journals). Is there a reason for this e.g. that you do not have access to the primary sources, or find them indigestible?

If you do need to rely on secondary sources, I'd suggest to focus on books and blogs whose authors are also producing the primary sources. Of the blogs you mention, I believe that Real Climate and Skeptical Science are largely authored by working climate scientists, whereas the others are not.

Of course a number of the blogs convey the message that climate science as a whole is utterly partisan and biased, so any output of climate scientists through secondary sources and summaries is untrustworthy. If you can't analyse the underlying primary evidence, and do not assign negligible prior probability to such a mass scientific conspiracy (or mass scientific error) then it is hard to refute that mindset. But you still have to ask who has the greater incentives here: is it really poorly paid scientists pushing a conspiracy or collective fantasy to get a bit more funding, or is it highly paid lobbyists, firms and commentators defending a trillion dollar industry, one which would be doomed by serious action on climate change?

Replies from: VipulNaik
comment by VipulNaik · 2014-07-07T22:36:47.749Z · LW(p) · GW(p)

As I mentioned in the post:

I read many papers from a diverse array of sources. I arrived at most papers either by clicking links on one of the blogs or websites mentioned above, or using Google or Google Scholar searches for specific topics. Any paper that I use as input to my opinion in a specific post will be explicitly linked in that post.

I can't include a list of papers right now because the list of papers will itself be determined in real time during my inquiries, but I will link to the ones I reference at each stage of research.

Of course a number of the blogs convey the message that climate science as a whole is utterly partisan and biased, so any output of climate scientists through secondary sources and summaries is untrustworthy. If you can't analyse the underlying primary evidence, and do not assign negligible prior probability to such a mass scientific conspiracy (or mass scientific error) then it is hard to refute that mindset. But you still have to ask who has the greater incentives here: is it really poorly paid scientists pushing a conspiracy or collective fantasy to get a bit more funding, or is it highly paid lobbyists, firms and commentators defending a trillion dollar industry, one which would be doomed by serious action on climate change?

This needs to be unpacked.

poorly paid scientists pushing a conspiracy or collective fantasy to get a bit more funding

Scientists are driven by a range of motives including research prestige and ego (those who have made statements in the past want to have those statements vindicated), the desire to impress and influence peers, etc. Getting more funding is part of the status game. These incentives can distort their findings, not necessarily forever but for long enough (just like the climate system, the system of scientific discovery is not in equilibrium; there are lags).

More importantly, simple cognitive and human biases can get in the way of proper analysis. One of the points I raise is that it's possible, based on Armstrong and Green's critique (I'm still investigating this) that climate scientists appear not to have consulted people in other relevant areas of expertise (specifically, forecasting and statistics). People in these areas of expertise have come up with a lot of relevant and counter-intuitive findings about how to go about this sort of tricky data analysis.

highly paid lobbyists, firms and commentators defending a trillion dollar industry

Of the sources I list, which ones do you think is written or funded by people who fit this description?

I believe that Real Climate and Skeptical Science are largely authored by working climate scientists

Judith Curry is also a working climate scientist.

Stephen McIntyre isn't a working climate scientist, but his criticism of Mann's statistical errors (which aren't necessarily relevant to the main arguments for AGW) have been acknowledged as essentially correct. I also took a reasonably detailed look at the specifics of the argument, and although I can't have very high confidence, I'm inclined to believe that McIntyre was right. He seems to be sufficiently rigorous in his work and a sufficiently strong skeptic that his critiques are worth reading. Moreover, he rarely claims more confidence than is warranted: he isn't publishing his own theories of climate change, he sees it as his job mainly to catch bad statistics.

Watts Up With That? seems to have poorer quality control in general, and often publishes opinion-type posts, but many of those opinion posts are guest posts by working climate scientists (albeit not necessarily high-profile ones). i wouldn't assign high credence to the blog.

Replies from: drnickbone, drnickbone
comment by drnickbone · 2014-07-08T07:24:21.000Z · LW(p) · GW(p)

Stephen McIntyre isn't a working climate scientist, but his criticism of Mann's statistical errors (which aren't necessarily relevant to the main arguments for AGW) have been acknowledged as essentially correct. I also took a reasonably detailed look at the specifics of the argument

Did you have a look at these responses? Or at Mann's book on the subject?

There are a number of points here, but the most compelling is that the statistical criticisms were simply irrelevant. Contrary to McIntyre and McKitrick's claims, the differences in principal component methodology make no difference to the proxy reconstructions. And the hockey stick graph has since been replicated dozens of times using multiple different proxies and methods, by multiple authors.

comment by drnickbone · 2014-07-07T23:52:08.442Z · LW(p) · GW(p)

On funding, it can be difficult to trace: see this article in Scientific American and the original paper plus the list of at least 91 climate counter-movement organisations, page 4, which have an annual income of over $900 million. A number of these organisations are known to have received funding by companies like Exxon and Koch Industries, though the recent trend appears to be more opaque funding through foundations and trusts.

On your particular sources, Climate Audit is on that list; also, from his Wikipedia bio it appears that Steve McIntyre was the founder and president of Northwest Explorations inc until it was taken over by CGX Resources to form the oil and gas exploration company CGX Energy Inc. He was a "strategic advisor" to CGX Energy at the time of his critique of the "hockey stick" in 2003.

Anthony Watts has received funding from the Heartland Institute#Affiliation_with_Heartland_Institute) which is also on the list. He claims it was not for the WUWT blog, and he approached them rather than the other way round.

Judith Curry has admitted to receiving "some funding from the fossil fuel industry" DeSmogBlog quoting Scientific American though again she claims no correlation with her views.