The Most Frequently Useful Thing

The most frequently useful thing I've gotten out of Overcoming Bias is not a technique or lesson so much as it is an attitude. It's the most ridiculously simple thing of all: to be in the habit of actually, seriously asking: is (this idea) really actually true? You can ask anyone if they think their beliefs are true, and they'll say yes, but it's another thing to know on a gut level that you could just be wrong, and for this to scare you, not in the sense of "O terror!--if my cherished belief were false, then I could not live!" but rather the sense of "O terror!--my cherished belief could be false, and if I'm not absurdly careful, I could live my whole life and not even know!"

Expecting Short Inferential Distances

One of many posts that gave me a distinct concept for something I previously had been only vaguely aware of, and this one kept coming back to me all the time. By now, I don’t think it’s an extreme exaggeration to say that I make use of this insight every time I communicate with someone, and of all the insights I picked up from OB, this might be the one I most frequently try to explain to others. It doesn’t seem like the most important thing, but for some reason, it immediately struck me as the most frequently useful one.

The Bottom Line.

Taking care, every time I think, to make sure I'm following out actual uncertainty or curiosity and am actually gathering evidence on what conclusion to write in, rather than rehearsing evidence or enjoying the sound of my own thoughts.

On the previous thread I mentioned the Mind-Projection fallacy and "the opposite of stupidity != intelligence" as being most frequently referenced, but on reflection, I think a reminder of the strictness of rationality makes the biggest difference in practice.

This passage from Technical Explanation sums it up: "But the deeper truth of Bayesianity is this: you cannot game the system. You cannot give a humble answer, nor a confident one. You must figure out exactly how much you anticipate the Sun rising tomorrow, and say that number. [...] You cannot do better except by guessing better and anticipating more precisely."

At this stage in my life, I find it easy to avoid to dogmaticism. False modesty, uncertainty, vagueness, and skepticism are all much more seductive. I work as a policy analyst in state government, and am frequently asked to provide forecasts. The dangers of narrow prediction intervals are well known, but I am tempted to be cautious and not focus my estimate. The field I work in is notoriously uncertain, but I can't do better just by being vague. Confidence and uncertainty have to be precisely balanced.

...the doctrine of non-reductionism is a confusion, rather than a way that things could be, but aren't. -EY, Excluding the Supernatural

Turned me into an atheist. Damn you.

The idea that that you shouldn't internally argue for or against things or propose solutions too soon is probably the most frequently useful thing. I sometimes catch myself arguing for or against something and then I think "No, I should really just ask the question."

Eliezer, I suspect you might find the answers to these questions less useful than you expect. The most useful things we've learned from you are probably going to be those things that we've already forgotten you wrote, because they've become a part of us -- because they've become background in how we live, how we think, and thus are completely invisible to us at any given time.

Most frequently useful - that my interest in being unbiased can become a sort of bias of its own, when I hear arguments from others, I can easily spot the biases, and I've worked hard to recognize that I have built-in biases as well that I can't discount.

A few ideas:

• the difference between Nobly Confessing One's Limitations and actually preparing to be wrong. I was pretty guilty of the former in the past. I think I'm probably still pretty guilty of it, but I am on active watch for it.

• the idea that one should update on every piece of evidence, however slightly. This is something that I "knew" without really understanding its implications. In particular, I used to habitually leave debates more sure of my position than when I went in---yet this can't possibly be right, unless my opposition were so inept as to argue against their own position. So there's one bad habit I've thrown out. I've gone from being "open-minded" enough to debate my position, to being actually capable of changing my position.

• That I should go with the majority opinion unless I have some actual reason to think I can do better. To be fair, on the matters where I actually had a vested interest, I followed this advice before receiving it; so perhaps this shouldn't be under 'useful' per se, although I've improved my predictions drastically by just parroting InTrade. (I don't bet on InTrade because I've never believed I could do better.)

• Sticking your neck out and making a prediction, so that you have the opportunity to say "OOPS" as soon as possible.

I think the most frequently useful thing I've learned is not in the content, but in the approach. The way of breaking things down, pulling together the themes, and looking at things in a way to makes the problems that seem to be there disappear. I guess that's what's been most frequently useful: when you get stuck, trying to see if you're looking at a problem in the wrong way, since often even if you're not it gets you unstuck.

The knowledge that communication with another brain through words and/or body language is hard. It's very lossy and almost always the source of error when I think someone has just said something absurd or incomprehensible. I may be ignorant of their train of thought, but that does not mean it's inherently random.

I use this constantly to quickly identify non-surface-level differing usages of terms, or to tell when I'm interpreting a phrase someone said differently than they mean me to. Latest concrete example: a couple hours ago, when I suggested that a D&D 4e melee was not well supported by the rules at all, and specifically that "what would the DM do?" summed up my objections. My roommate replied that he would do exactly what he always did, which didn't jive with what I (thought I) was saying, and I immediately knew we were interpreting "melee" in different ways.

Either that's availability bias, or it comes up very frequently, since the most recent event was mere hours ago. :)

Taboo Your Words and Wrong Questions together. There are numerous debates over definitions that I'd previously have happily engaged in, but now I just take a glance at them and shrug them off as trivially solvable by tabooing. Applied this into just about everything, from philosophy and Searle's Chinese Room to politics and "is online piracy theft". I feel it has considerably clarified my thinking (and it was my second candidate to the "most useful thing" thread).

Shut up and do the impossible, or lesser levels of 'dont run away from the problem'

In particular, the AI box experiment applied to persuasion in general. I make a lot more progress in argument when I focus on changing the persons mind, rather than picking apart their flawed arguments or defending mine

This one comes up in philosophical discussions of ethics all the time: Sorting Pebbles into Correct Heaps

I end up referring to right vs right' a lot. Especially w.r.t. ethical naturalism. Though now that I think of it, the actual content there might have been in a follow-up post.

Most frequent would have to go to my avoidance of settling with cached thoughts. I notice, revise, and completely discard conclusions much more regularly and effectively when I recognize the conclusion was generated as soon as a question was asked.

reversed stupidity

followed by

dissolving the question and mind-projection fallacy.

Asking myself “Couldn't I get Y without X?” whenever I'm considering doing expensive/time-consuming/non-fun action X in order to get benefit Y.

...is also the most important thing, though that wasn't necessarily the case.

I find that over the last year or so, whenever a debate ceases to be about something in the world and becomes about pure semantics, a little bulb flashes up in my head - Standard Debate! "Yes, I know it's only tiny, but the question I'm asking is, is Pluto a planet?"

Inferential distances. Most frequently when calibrating the best way to explain something. More drastically on the few occasions I lacked the cached thoughts to absorb what I was hearing.

The most frequently useful thing I have learned from OB is to update assumptions based on new information on an ongoing basis. I think this idea ties in nicely with that of standing against maturity, if maturity is taken to mean a certain rigidity, an inflexibility of purpose and outlook.

The most frequently useful thing I've learned is to be much more distrustful of my motives, my reasoning, my judgments, and my capacity for self-understanding and being aware of what is going on in my head (and why).

The series on free will, probably, because it short-circuits a lot of fruitless contemplation of the meaning of life and such.

deleted

How to take Joy In The Merely Real.

The most frequently useful is map and territory. It is something that I deploy on a daily level to remind myself of the value of never believing that I know everything, and to leave myself open constantly to the possibility of making mistakes and updating my beliefs. It is also one of the most basic things I teach at Intentional Insights.

the informal use of the bayesian probability method to analyze the plausibility of various statements...

Updating my beliefs based on new information. This has helped me to move away from emotional defensiveness and toward intellectual curiosity when I'm confronted with new information that is contrary to my beliefs or desires.

Reminding myself that "I want to become stronger" emboldens me to make rational choices that require more courage than usual. This has caused me to take a class on entrepreneurship, for example, when otherwise I may have felt too shy to do so.

Noticing subtle confusion.

Applying this skill has consistently led me to surprising and useful information. So far the gains seemed to have grown with the degree of subtleness I can recognize.

Mostly things that I've observed. For instance:

• If you try to anticipate someone else's misinterpretations of what you say, you are likely to match their reply to your expectations using regular-expression-like matching and reject their response, without fully parsing it.

• Often, it's better to say one insightful thing than to say one insightful thing and two less-insightful things, because the less-insightful things are easier to respond to.

• Developing expertise in overcoming bias, and "trying really hard" to overcome bias, doesn't overcome bias. It can make bias worse, by becoming an excuse not to update in response to the ideas of others.

I had already been a Bayesian and fan of Kahneman-Tversky for 15 years before I started reading OB. So I didn't learn the basics there.

Given that, I'd have to say the most important thing I learned was to exercise unflagging discipline when thinking about values. For me, this means (a) remember to keep track of terminal versus instrumental values in any problem framing and (b) realize that most people's terminal personal values are complex and != "maximize pleasure".

I tend to agree with MBlume - the most frequently used principles are probably assimilated too well. But let's see... the Bayesian worldview in general made me much more interested in probability, making me take the most "mathy" probability course in Uni early on and to plan on reading Jaynes and Pearl within the next half a year. Maybe it was The Dilemma: Science or Bayes that clinched the deal?

Skimming the list - Mind Projection Fallacy, Nobody Knows What Science Doesn't Know and Science as Attire often come to mind in contexts of what other people do wrong (I quoted the first and second principles in a few discussions). Making Beliefs Pay Rent, Mysterious Answers to Mysterious Questions, Making History Available, Cached Thoughts, Bind Yourself to Reality - I try to apply to myself on a regular basis. In my professional capacity, I try to apply (not very successfully at the moment) The Planning Fallacy and Hold Off Postponing Solutions.

The Robber's Cave Experiment was the post that got me hooked on OB in the first place. I have cited it many times. Finally, the posts on morality are frequently used in the sense that I refer myself to them every time moral discussions crop up.

I also think you should include other writings. The list you gave in the previous post does not include Robin's articles (obviously), but he certainly did leave a mark as well. (To mind come his posts on medical spending and Politics is not about Policy, but undoubtedly there are many more.)

But this is already a large list. Perhaps a series is in order, with the first book being "A Gentle Introduction".

for me, guessing the teacher's password was a real 'oh wow' moment for me that changed the way i went about learning and approaching my classes. it definitely awoke an awareness that i, myself, was simply guessing the password most of the time and it illuminated that this was the way the lecture system primarily functioned in my classes. as a result, i feel i mentally 'engage' more with the material presented and think critically not only about the material itself but on the meta-level of how and why the material is being presented in the way it is.

this was most apparent in an abnormal psychology class where we were presented with case studies containing the story of an individual and then meant to diagnose that person according to DSM IV criteria. you might read about an eccentric individual who dresses in eccentric fashions, has occasional muscle tremors, and will ramble incoherently, jumping from topic to topic. "aha! schizophrenia, disorganized type!" and that was that.

there was no follow up on exactly what that meant. there was simply a bucket of symptoms with a label on it and lo-and-behold this person had a cluster of symptoms from that bucket.

though schizophrenia may be a poor example due to science's weak understanding of it, this process was repeated across the board regardless of the mental ailment. this may simply have been due to the broad nature of the class and time constraints but i remember coming away with nothing but a bunch of labels and little understanding about what was actually happening.

link: http://www.overcomingbias.com/2007/08/guessing-the-te.html

In "teaching" situations with adults I often try to impart the modesty argument. I want my audience to become less sure of themselves, less sure of their judgements, less sure of their brains, less ready to disagree and thus more open to suggestions.This is a theme that repeats itself many times. Progress is rather uncertain so maybe this isn't very useful! Darn! Perhaps I should change course and go onto signalling and status issues instead, so we can arrive at the truth about ourselves.

Generalising from one example.

"I notice that I am confused." That phrase pops into my head unbidden at least once a week now.