The Scales of Justice, the Notebook of Rationality

This two-side bias appears to fit in nicely with the neuroscience of decisionmaking where anticipatory affect appears to be weighed together to decide wheter an action or option is "good enough" to act on. For example, in http://sds.hss.cmu.edu/media/pdfs/Loewenstein/knutsonetal_NeuralPredictors.pdf there seems to be an integration of positive reward in the nucleus accumbens linked to the value of the product and negative affect related to the price in the insula, while and medial prefrontal cortex apparently tracks the difference between them.

There is definitely room for a more complex decision system based on this kind of anticipatory emotional integration, since there might be more emotions than just good/bad - maybe some aspects of a choice could trigger curiosity (resulting in further information gathering), aggression (perhaps when the potential loss becomes very high and personal) or qualitative tradeoffs between different emotions. And the prefrontal cortex could jump between considering different options and check if any gains enough support to be acted upon, returning to cycle if none seem to get quite the clearcut support it ought to.

This makes a lot of sense from a neuroscience perspective, but as an approximation to rationality it is of course a total kludge.

I believe that there was something about a similar approach in a paper "Risk at a Turning Point?" by Andrew Stirling. He argued that analysis of risk should group all the risks as a vector valued quantity, rather than a scalar. That should be just a valid in this more general context: risks, costs and opportunities of a particular scenario can then be represented on a big vector, and each interest group applies their own method to bring it down to a scalar value (or probablility distribution) along the "support/oppose" continuum.

Andrew was focusing on the fact that generally the one to do the estimate was a government or a corporation that would apply their own method to get from the vector to the scalar, and only the scalar was announced. If the full vector was announced, however, it was easier for groups with different values to come up with their own estimate of the scalar "support/oppose" distribution. As well, they could easily add extra elements to the vector (things like "the project is an eyesore"), and see how that changed their estimate, rather than adding it as an extra and having those fruitless "the project is an eyesore" vs "yes, but it'll bring in cash" debates.

The vector could be what little ol' dame rationality writes down in her notebook.

Keep in mind that in many situations we do in fact have to make a binary decision between two alternatives. Often it reduces to a go/no-go decision. In that case this heuristic of reducing multi-valued vectors to a single scalar weighting factor is a necessary step.

Hal, even on binary decisions, the affect heuristic still leads to double-counting the evidence. If being told that the plant produces less waste causes us to feel, factually incorrectly, that the plant is less likely to melt down, then the same argument is being counted as two weighting factors instead of one.

But, by thew same token, wouldn't then being told that the reactor is more likely to meltdown lead people to think it produces more waste. If I multiply the true effects of everything by 10, wy will tht affet the binary choice?

In response to the statement, "If you tell people a reactor design produces less waste, they rate its probability of meltdown as lower", this may be the result of a useful heuristic if technologies generally improve overall. Consider computers: if I asked people to guess if the amount of memory in a desktop computer with a 300MHz processor is less than or greater than that in a system with a 2GHz processor, they might reason that the computer with the faster processor is newer, that both technologies have improved, and the 2GHz system most likely has more memory as well. Similarly in the example, people may think that both anti-meltdown and anti-waste technologies are likely to have improved concurrently. This isn't to say that both factors don't need to be looked at separately in the "real world" - only that I'm not sure how we could consider any other answer rational in the absence of further information.

Basically, I'm curious if benefits and costs are really positively correlated to one another in the real world, as shown in Exhibit 1 in the PDF.

this may be the result of a useful heuristic

Another heuristic may be our habit of expecting some sources - say, newspapers - to present the arguments pro and agaisnt the issue ("this will clean up the beach, but costs money"). If they say "this will produce more waste" and leave it at that, we may assume that's the only way the reactor is different.

Great post. You're on a roll, Eliezer. Hal, I query how often the best decision-making process really is binary go/non-go. That humans often reduce a decision making instant to go/non-go "as an approximation to rationality it is of course a total kludge" seems plausible to me (to use Anders Sandburg's words).

Eliezer, I doubt the justice system's guilty/not guilty approach is grounded in rationality either. "If Justitia must consider any more complex issue, she should relinquish her scales or relinquish her sword" -and I think the underlying issues regarding "justice" are almost always more complex. But then again, I think the approach with justice should be economic, incentive-based, and empirically grounded, rather than punitive and grounded in social norms the way it is now (in the U.S.).

Anders, is there any research on the degree to which human predisposition to "treat discussion as a form of combat, an extension of war" reduced to 2 binary and oppositional players is grounded in a primate aesthetic. The parallel to primate researcher discussions of alpha and challenger males seems strong to me.

My guess is that nuclear waste production and chance of reactor meltdown are very weakly correlated. Both are decreased if the reactor is designed by a particularly conscientious group of researchers.

To be clear: I’m not arguing against. I’m asking to clarify. I find myself thoroughly confused by this article.

How is a higher probability for meltdown NOT a “point against” the reactor—and how is less waste NOT a “point for?” I think I’m missing some underlying principle here.

If you tell people a reactor design produces less waste, they rate its probability of meltdown as lower.

Wait. WHAT? How does that even make sense?

I suppose if you gave me a long boring lecture about reactors, and then quizzed me on it before I remembered the facts (with my house cat memory), I would could get this wrong for the exact reasons you described, without being irrational.

Suppose there’s a multiple choice question, “How much waste does reactor 1 produce?” and I know that reactor 1 is the best across most categories (has the most points in its favor), and I know that all reactors produce between 10 and 15 units of waste, then my answer would be (b) below:

(a) 8 units

(b) 10 units

(c) 12 units

(d) 14 units

And of course, there’s every possibility that “reactor 1” didn’t get the best score in waste production. Didn’t I just make the same mistake as Eliezer described, for completely logical reasons (maximum likelihood guess under uncertainty)? This isn’t a failure of my logic; it’s a failure of my memory.

In real life, if I expected a quiz like this, I would have STUDIED.

Why else would anyone expect an overall-best-ranking reactor to necessarily be the best at waste production?

Here’s another idea. Suppose that long boring hypothetical lecture were on top of that so confusing that the listener carries away a message that “a meltdown is when a reactor has produced more waste than its capacity.” Then it is a perfectly logical chain of reasoning that if a reactor produces less waste, then its probability of meltdown as lower. But this is poor communication, not poor reasoning.

The paper "The Affect Heuristic in Judgments of Risks and Benefits" doesn't mention explicitly separating benefit from risk in the critical second experiment (and probably not the first either, which I didn't read). If I were brought in and given the question, 'In general, how beneficial do you consider the use of X in the U.S. as a whole?', then I would weigh all positive and negative aspects together to get a final judgment on whether or not it's worth using. "Benefit" CAN be a distinct concept from risks, but language is messy, and it can be (and I would) interpreted as "sufficiency to employ". As a result, depending on the reader's interpretation of "benefit", it's possible that any lowering of perceived risk will NECESSARILY increase perceived benefit, no logical error required.

Rather sloppy science, if you ask me.

Cool.

Lady Rationality... love it. I think I want her as a tattoo.

Another problem with policies like this hypothetical nuclear reactor is that people don't have access to facts about the future or hypothetical futures, so we're left with estimates. People don't acknowledge that their "facts" are actually estimates, and don't share how they're estimating things. If we did this, politics would be better. Just give your methods and assumptions, as well as the estimates that are determined by the methods and assumptions, of costs and benefits, and then people can pick the choice(s) with the best estimated benefits - costs.

The other thing about political arguments is that people don't start with the foundation to the above, which is values. People will often talk about "lives saved", which is ridiculous, because you can't save lives, only postpone deaths. People who don't agree on values aren't ready to look at estimated costs and benefits. If I value 3 years of postponed-death at $100k, and someone else values it at $10, then we're almost certain to disagree about which policies are best. Values are the prices at which you trade things you like/want/value.