Cached Selves

Another technique: thought quarantine. New ideas should have to endure an observation period and careful testing before they enter your repertoire, no matter how convincing they seem. If you adopt them too quickly, you risk becoming attached to them before you have a chance to notice their flaws.

I've found this to be particularly important with Eliezer's posts on OB and here. Robin Hanson's and most other posts are straightforward: they present data and then an interpretation. Eliezer's writing style also communicates what it feels like to believe his interpretation. The result is that after reading one of Eliezer's posts, my mind acts as though I believed what he was saying during the time I spend reading it. If I don't suspend judgment on Eliezer's ideas for a day or two while I consider them and their counter-arguments, they make themselves at home in my mind with inappropriate ease.

I won't go so far as to accuse Eliezer of practicing the dark arts--I agree that communicating experiences is worthwhile. I read OB for the quality of the prose as well as idea content, both of which stand out among blogs. But while this effect hasn't been articulated in comments as far as I know, I suspect that it contributes to the objections Robin Hanson and others have to Eliezer's style.

We left out one strategy because we didn't have scientific support for it. But introspectively:

3. Change or weaken your brain’s notion of “consistent”. Your brain has to be using prediction and classification methods in order to generate “consistent” behavior, and these can be hacked.

• 3f. Your brain remembers which "simple" predictor best described your decision, so change the pool of predictors for describing your decisions that your brain counts as "simple".
• • 3fi. Learn to judge yourself, not by the best inference or decision you could have made in hindsight, but by the best inference or decision you could realistically have made at the time. This way, poorly-informed or impaired past decisions are evidence of your past poor information or impairment, and are not evidence that it is hopeless to try to make a better decision now that you have better information or are less impaired. To help your brain count this standard of judgment as "simple", you may wish to make a habit of judging everyone this way.
• • 3fii. Your brain learns to predict other peoples' judgments by learning which systems of predictive categories other people count as "natural". If you have to predict other peoples' judgments a lot, your brain starts to count their predictive categories as "natural". The effect can be viral (especially with categories whose social definitions tacitly refer to Schelling focal points, pooling equilibria, or punishment of non-punishers, with a penalty for disagreeing about the boundary), and it can change how you think about yourself. Try to control social access to your brain's pool of "simple", "natural" predictive categories, and try to unpack category definitions so that when they are not simple, your brain sees how. (Or try to live so that your most intense experiences of thinking only make predictions of physical consequences, and not predictions of other peoples' judgments.)

On this subject: Autistic spectrum conditions and obtuse concrete literalism are sometimes a good temporary defense against other peoples' unnatural category systems, but you should have a backup plan.

Everybody's writing about removing this effect in the future. But how about in the past?

How much of your present day self concept is delusional (or at least arbitrary)?

I liked this article, though I might change my mind about that. I’m someone who changes my opinions based on conversation and reasoning.

Great post.

Here's some additional reading that supports your argument:

Distract yourself. You're more honest about your actions when you can't exert the mental energies necessary to rationalize your actions.

And the (subconcious) desire to avoid appearing hypocritical is a huge motivator.

I've noticed this in myself often. I faithfully watched LOST through the third season, explaining to my friends who had lost interest around the first season that it was, in fact, an awesome show. And then I realized it kind of sucked.

This is a great post, especially all of the technique suggestions.

Socially Required Token Disagreement: I'm especially surprised by the "drive safely" study - and it's especially weird since "keeping America beautiful" would seem to contradict putting a big ugly sign on your front lawn. Maybe the effect wasn't through the person's support for vague feel-good propositions, but through their changed attitude to following requests by strangers knocking on their door.

2b. Positive hypocrisy. Speak and act like the person you wish you were, in hopes that you’ll come to be them. (Apparently this works.)

This does work. I found that when I noticed I was quiet and didn't talk to people often, that I didn't like being that way. I wanted to reach out. It took four years to break the habit, but now my friends know me to be a generally "out spoken and outgoing" person. In other words, I had an image of what I wanted to be (more outgoing) and thought of what would an outgoing person do (talk to the person sitting next to me in class, get involved in a club, etc) and tried to slowly integrate those things into my life. I think the main point though, is that a person has to honestly want that image of themselves to become an actuality, otherwise the efforts end up falling short.

This was a very interesting post. I'm glad I saw it- it explains behaviors I have exhibited, and the behaviors of friends etc... I will definitely try to be more aware of this.

This is something I've noticed as a factor in my own behavior since I was a child. I never tried isolating myself from social influences to avoid pressure on my personality though, rather, at an early age, I consciously rejected the idea that I had a distinct "true" personality that existed irrespective of circumstance. My strategies mainly focused on creating my own pressures to conform to so that I could shape my contextual behavior in directions I wanted.

My first top level post is actually an example of this, since I created it not just to provoke discussion or enlighten, but to unpack my sense of humor. I tend to be happier in social circles where I express a sense of humor than ones where I don't, but while I could act like that whenever I want to, in practice, if I haven't set up a persona that it's consistent with, I usually don't.

I also tend to avoid anonymity online, so that I can use others' expectations of me to reinforce standards for my own behavior. I use the same online handle pretty much everywhere I go, so that I can enforce some continuity between my online personas, and so that my reputation everywhere on the internet rests on my behavior anywhere, and I can keep myself from descending into GIFT

Whenever you are about to make a decision which you do not care much about, one way or the other, use one of the following algorithms:

• Flip a coin (or some other external randomized choice), and base your decision on the result
• Explicitly avoid the choice(s) you made last time

Consistency effects seem to me to be the sort of error that more intelligent people might be MORE prone to, and is thus, it seems to me, particularly important to flag.

BTW, 3E seems to me to be by far the most important of the rationality suggestions given, largely because it actually seems practical.

Could more people please share data on how one of the above techniques, or some other technique for reducing consistency pressures, has actually helped their rationality? Or how such a technique has harmed their rationality, or has just been a waste of time? The techniques list is just a list of guesses, and while I'm planning on using more of them than I have been using... it would be nice to have even anecdotal data on what helps and doesn't help.

For example, many of you write anonymously; what effects do you notice from doing so?

Or what thoughts do you have regarding Michael Vassar's suggestion to practice lying?

How can you tell whether one's self might be getting hijacked or if it's getting rescued from a past hijacking?

E.g. I've been a long-time OB reader but took a couple of months off (part of a broader tactic to free myself of a possible RSS info addiction, and also to build some more connections with local people & issues via Twitter). I brought OB back into my daily reading list last week, read a few of Robin's posts and wondered where Eliezer was at...

Now I find myself here at LW, articulating thoughts to myself as I read and catch up, feeling impelled to comment... After a couple of hours I find I'm saying to myself that this is really great and I should rearrange my daily routine yet again -- maybe cut down on the Twitter use -- to spend regular time here becoming more rational.

So: hijacked by LW or rescued from Twitter? Are there any objective measures that could be used?

(I'm relating a personal experience but I don't want to give the impression I'm just looking for help with this particular situation. I'm wondering more generally.)

In a follow-up, Jonathan Freedman used threats to convince 7- to 9-year old boys not to play with an attractive, battery-operated robot. He also told each boy that such play was “wrong”.

Along the lines of 3a, I consider the word 'wrong', and all variations, as if they were uttered at gunpoint. From my own observation the difference between the two is little more than degree. Describing (usually internally) such social influences in terms of raw power allows them to be navigated without having my self concept warped into undesirable patterns or riddled with undesired natural categories.

This reminds me heavily of some studies I've read about pathological cases involving, e.g., split-brain patients or those with right-hemisphere injuries, wherein the patient will rationalize things they have no concious control over. For instance, the phenomenon of Anosognosia as mentioned in this Less Wrong post.

The most parsimonious hypothesis seems, to me at least, that long-term memory uses extremely lossy compression, recording only rough sketches of experience and action, and that causal relationships and motivations are actually reconstructed on the fly by the rationalization systems of the brain. This also fits well with this post about changing opinions on Overcoming Bias.

I think I have seen a similar argument made in a research paper on cognitive neuroscience or some related field, but I can't seem to find it.

As someone (Heinlein, I think?) said: "Man is not a rational animal, he is a rationalizing animal."

Oh man. I already knew about this effect when I spent the summer in Atlanta. But I am not very good under social pressure, so when all the panhandling gentlemen recognized my clueless wandering around the streets and started demanding a dollar or bus fare, I felt like I had to give it to them. (I did this until I ran out of cash, and then I just said "Sorry! No cash! I need to catch the bus! Bye!") One guy asked for 80 cents for water from the vending machine, so I offered him my water. To which he promptly replied that he would like my water in addition to 80 cents because, "well, I live in a shelter." It's true! Kind of! I don't live in a shelter! Anyways, I am concerned that this experience has turned me more liberal than I would be otherwise. Even though I can sort of tell that I shouldn't be. Or rather, that it shouldn't be from this experience. Social pressure be damned!

So. How do I unbias myself? =]

For example, if I hang out with a bunch of Green Sky-ers, and I make small remarks that accord with the Green Sky position so that they’ll like me, I’m liable to end up a Green Sky-er myself.

Courtesy reference: A Fable of Science and Politics describes a conflict between two fictional underground factions, who were in disagreement regarding the appearance of the sky, long lost to them.

This post seems to imply that the self-consistency bias is irrational, but it doesn't argue strongly that becoming more self-inconsistent leads to better outcomes. In fact, it hints that the self-consistency bias is strong and natural, which would suggest that it might be beneficial in the EEA.

For example, it may be more beneficial to consistently carry through plans than to switch to the best-appearing alternative at every step.

Another idea: Possibly the "incorporate small decisions into one's self-concept" is a way of seeking individuation; being an unusual person may be advantageous in some way.

Agree with and like the post. Two related avenues for application:

1. Using this effect to accelerate one's own behavior modification by making commitments in the direction of the type of person one wants to become. (e.g. donating even small amounts to SIAI to view oneself as rationally altruistic, speaking in favor of weight loss as a way to achieve weight loss goals, etc.). Obviously this would need to be used cautiously to avoid cementing sub-optimal goals.

2. Memetics: Applying these techniques on others may help them adopt your goals without your needing to explicitly push them too hard. Again, caution and foresight advisable.

This effect is interesting from the rationalism perspective because it has three separate effects: (1) making us believe false things about our past mental states (or even false things about the world); (2) creating a disconnect between why we claim/believe we are saying or doing something, and why we actually are; and (3) the change in our behaviors/desires themselves. While (1) and (2) clearly represent decreases in rationality/sanity, what can we say about (3)? Don't we all believe Hume around here?

These consistency effects are reminiscent of Yvain’s large, unnoticed priming effects -- except that they’re based on your actions rather than your sense-perceptions,

If they wanted to show that, they should have had a control group that observed other people taking those actions.

Observing yourself doing something is still observing it, and priming could still account for the results.

I think this is a great post.

Really made me think about how this might apply to me, and I've already decided to make a few changes based on some of your suggestions (mostly in how I phrase things when describing myself).

As social creatures, I wonder if the effect is stronger when these consistency effects rise up in group situations. Does our brain try harder to stay consistent with an identity that makes us part of a group rather than an "individual" identity?

This certainly could explain a few things about how solid political/tribal/religious identity is, not to mention the most intense kinds of metal-heads, comic book geeks, browncoats, free software evangelists, etc (all types of hardcore fans that build their identity around what they happen to enjoy).

The unsettling part comes next; Freedman and Fraser wanted to know how apparently unrelated the consistency prompt could be. So, with a third group of homeowners, they had a “volunteer” for an ostensibly unrelated non-profit ask the homeowners to sign a petition to “keep America beautiful”. The petition was innocuous enough that nearly everyone signed it. And two weeks later, when the original guy came by with the big, ugly signs, nearly half of the homeowners said yes -- a significant boost above the 19% baseline rate. Notice that the “keep America beautiful” petition that prompted these effects was: (a) a tiny and un-memorable choice; (b) on an apparently unrelated issue (“keeping America beautiful” vs. “driving safely”); and (c) two weeks before the second “volunteer”’s sign request (so we are observing medium-term attitude change from a single, brief interaction).

I'm strongly tempted to defy the data.

I am working on my own time to understand how to apply Value of Information Analysis to the projects I handle for an oil company, particularly regarding geoscientific matters of reservoir characterization.

This is a really good post. I particularly like the suggestion that we don't have to infer and cache conclusions about ourselves when we screw up and don't return a library book. (Of course, other people would be rational to cache a conclusion about us because thinking differently wouldn't be a self-fulfilling prophecy.)

I must be falling to the dark side because I read this and thought "so this is how I can convince people of things: give them a dollar to say they agree with me."

3e sounds a lot like narrative therapy. If you're interested in that method, reading more about narrative therapy could help.

More evidence that social work and LW have many similar aims, and methods that can be used for both.

Although in this post the authors have emphasised more on the hijacking when we say things I find it somehow related to the "cognitive costs of doing things" ( see http://lesswrong.com/lw/5in/the_cognitive_costs_to_doing_things/ ) When you do something you always pay a price. Maybe in the list of what to do could be added to be prepared and estimate the price you are going to pay when you are planning to do something.

Also beware of the "beauty bias" (see www.overcomingbias.com) if a handsome/beautiful person tells you something its seems you are more likely to agree with him/her.

One thing I would point out is that the arguments presented here are a considerable effort into the examination of one’s own personal psyche, and that of the common psyche.

While it can be a definite benefit to examine this topic, I advise caution of moderation in the attempt.

I admire the authors own example in doing the equivalent: "I’m not recommending these, just putting them out there for consideration"

My main point of argument is that examination need not be experimentation, we can form hypothesis for consideration and not be burdened with the responsibility of an incorrect interaction.

I find the examples presented in this argument unnecessary (Freedman) if the examiner is capable of limited self examination.

In consideration of the main argument I would say that in my own experience it is possible to advise some of their own awareness without adhering to the presented guideline, that others may be of a nature above the need for any guideline, and, others yet may find completion in never knowing the presented guideline.

By listing those "suggestions," you are causing people at least one person to try to use them even though they are in my judgment largely worthless or at least not worth the time and effort required to try to adopt them (this judgment means little compared to actual evidence of their relative effectiveness, but since I haven't seen any it will have to suffice as a prior). I have also seen no plausible argument here that this type of bias actually causes unhappiness, and so I therefore care nothing about it.

The cache problem is worst for language because its usually made entirely of cache. Most words/phrases are understood by example instead of reading a dictionary or thinking of your own definitions. I'll give an example of a phrase most people have an incorrect cache for. Then I'll try to cause your cache of that phrase to be updated by making you think about something relevant to the phrase which is not in most peoples' cache of it. Its something which, by definition, should be included but for other reasons will usually not be included.

"Affirmative action" means for certain categories including religion and race, those who tend to be discriminated against are given preference when the choices are approximately equal.

Most people have caches for common races and religions, especially about black people in USA because of the history of slavery in USA. Higher quantity of relevant events gets more cache. More cache makes it harder to define.

One who thinks one acts in affirmative action ways for religion would usually redefine "affirmative action" when they sneeze and instead of hearing "God bless you" they hear "Devil bless you. I hope you don't discriminate against devil worshippers." Usually the definition is updated to end with "except for devil worshippers" and/or an exclusion is added to the cache. Then, one may consider previous incorrect uses of the phrase "affirmative action". The cache did not mean what they thought it meant.

We should distrust all language until we convert it from cache to definitions.

Language usually is not verified and stays as cache. It appears to be low pressure because no pressure is remembered. Its expected to always be cache. Its experienced as high pressure when one chooses a different definition. High pressure is what causes us to reevaluate our beliefs, and with language, reevaluating our beliefs leads to high pressure. With language, neither of those things tends to be first so neither happens usually. Many things are that way but it applies to language the most.

Example of changing cache to definition resulting in high pressure to change back to cache: Using the same words for both sides of a war regardless of which side your country is on can be the result of defining those words. A common belief is soldiers should be respected and enemy combatants deserve what they get. Language is full of stateful words like those. If you think in stateful words, then the cost of learning is multiplied by the number of states at each branch in your thinking. If you don't convert cache to definition (to verify later caches of the same idea), then such trees of assumptions and contexts are not verified, which merge with other such trees and form a tangled mess of exceptions to every rule which eventually prevents you from defining something based on those caches. That's why most people think its impossible to have no contradictions in your mind, which is why they choose to believe new things which they know have unsolvable contradictions.