Singlethink

Jeez, what's next? "Optimism Bias and My First Kiss"?

As an avid reader, I have to say I'm enjoying the personal revelations. They have much in common with certain varieties of religious exuberance. In fact the whole sequence has the feel of an Olde Tyme Rationalist Revival.

But butt smacking recollections from summer camp? Surely there has to be a line somewhere.

I know. I can't help it. That was my first step into advanced rationality. I mean, what was I supposed to say? It was an important point and I wanted to get it across.

The truth doesn't always save a person. It especially doesn't always save a person from embarrassment.

This sort of thing intrigues me. It underlines for me that our "technology of the mind" is still very young and unstructured.

Imagine what would happen if this one skill were explicitly taught in schools?

I prefer to see examples like this. There are lots of good reasons for grounding these discussions in the concrete.

An exercise I adopted when I was a child that I did not read anywhere is to recall or reconstruct the thought that led to my current thought, then the thought before that, etc. By examining many such transitions, I discovered some generalizations about the unconscious function that takes thought N to thought N+1.

Did you ever actually assert that your reasoning was out of principle, only to suddenly remember the experience through an unrelated pathway and thus the remember the real reason of your refusal later on, contradicting that initial assertion, or was it simply that you had never actually considered your reason of refusal until you were 15, and that it had only felt like you had been making the assumption that you had refused out of principle when you had never asserted that as truth- you had just never considered your reason, and you knew when you were 15 that there was reason to refuse- because it was a negative-sum game.

So, Eliezer. You're interested in the Singularity. And the Singularity will come about by means of self-improving artificial intelligence. If I understand correctly. Meanwhile, you're also interested in self-improvement of the human wetware - hence your contributions to this website. There's an obvious similarity between these two interests, possibly even an overlap.

Took me a while to notice it.

Richard,

By examining many such transitions, I discovered some generalizations about the unconscious function that takes thought N to thought N+1.

I'm very interested, please elaborate!

Eliezer,

You started noticing the value of beating specific biases at age 15, but apparently didn't get the importance of heuristics and biases in general till 2003. Why did that reflective generalisation take so long, when in hindsight it's so vital for your chosen path? That's not meant as criticism - I've done this too. I want to know how to fix it.

Incentive to think about this: doing so might lead you to notice something vital for safe seed AI.

RI, the problem was simply the size of the planet and its scientific literature. I just didn't run into the field of heuristics and biases until that late. I realized it was important the first time I read a mention-in-passing on a webpage - though I didn't realize how much there was to know and how little of it I knew; Emil Gilliam finally mailed me a copy of "Judgment Under Uncertainty".

At the age of fifteen, I realized the importance of defeating biases in general, I just didn't know there was a field that studied them. I knew the importance of rationality in general, but I'd never heard the word "Bayesian". It's a frighteningly big world out there - not like a hunter-gatherer tribe, where you can presume you know everything anyone else knows.

-- and since most writings in psychology are worthless, it is easy to give up on the whole field before one discovers worthwhile writings like "Judgement Under Uncertainty" and "The Moral Animal".

RI, a large part of my motivatation was simply to practice a mental skill: it is a delightful feeling to improve drastically one's ability to observe one's own deliberations. Three decades and a severe bump on the head separate my teenage years from today, and today I am almost completely unable to do this exercise.

BTW, it is my guess that the exercises Eliezer and I describe will confer most of their benefits on exercisers who are still teenagers.

RI, to answer your question: the function that takes thought N into thought N+1 is complex enough that I did not learn anything that could be put into neat sentences, nor do I retain any declarative memories of what I learned except that the deliberation proceeded in a much more "predictable-in-retrospect" manner when I thought about some themes than when I thought about others. E.g., I remember that thinking about my mom produced very opaque chain of thoughts.

The practice Eliezer describes strikes me as of greater potential benefit than the one I describe, but perhaps the one I describe can be accomplished by a greater fraction of teenagers reading these words. Very few individuals are blessed with the delightful hardware that the teenage Eliezer had available for such exercises.

Thanks for your replies,

and since most writings in psychology are worthless, it is easy to give up on the whole field

Fair point. But if it annoys us, think how the rational minds in the field must feel!

...it is a delightful feeling... ...did not learn anything that could be put into neat sentences... ...much more "predictable-in-retrospect"...

Are you sure the model that your mind was using to report its own thought process corresponded to what was really going on under the bonnet? I've barely scratched the surface yet in studying bias and rationality (the 2 books you recommended are queuing on my shelf) but a recurring theme seems to be that introspection about one's own thought processes is often demonstrably unreliable, even among highly intelligent subjects working within their fields of expertise and reporting very high confidence in their introspections.

Eliezer,

Maybe I'm just biased towards cross-fertilization between disciplines. I was just wondering about other vital but relatively obscure knowledge being missed for similar reasons, and if there might be a way to attack the problem. Anyhow, please thank Emil for us, a lot of people are getting stronger thanks to her little gift! :-)

...her little gift...

That's what I get for posting at 1am. Apologies to Emil, who I see is an SIAI donor and therefore one of my favorite people :-)

That's a horrible game. Did you have a name for the game as well? Spank-to-spank?

I remember playing a game in which we would take turns punching each other in the stomach as hard as we could while trying not to flinch, but I viewed it as positive sum. Even though you get punched it's still a lot of fun.

Interesting. As a child I thought I could remember everything, so lying was easy, it's own memory of having lied, and thus distinct from reality. It was only much later that I realized it was even possible for the two to become horribly confused in one's own mind. But did you lie and forget the lie? Or did you just incompletely remember and add the part about negative sums as an explanation of the behavior later on? I think the "sweeping" of our minds' corners is also prone to give us false memories. We think, "well, why did I do that? Oh yeah! Now I remember..." You might have just been lucky to stumble upon what you now believe to be the real memory, but introspection, especially of the past isn't always so accurate.

Are you sure the model that your mind was using to report its own thought process corresponded to what was really going on under the bonnet?

I was very confident that I could reliably retain or reconstruct the last five or so actual thoughts of my conscious deliberation -- and the actual ordering or sequencing of those thoughts. Although recalling or reconstructing a thought sometimes required a few seconds of effort or patience, the reconstruction sure felt like a reliable observation rather than an interpretation or the updating of a model or the use of a learned model. As for extracting generalizations or regularities from those observations, like I said, I can recall making only one (declarative) generalization, but I have no reason to believe that I could not bring the same regularity-noticing, causality-extracting powers to bear on those observations that I could to any observations made by my senses. (When observing a very complex process, the regularity-noticing consists mostly of implicit learning rather than explicit, declarative learning -- or so it seems to me.)

Interesting. Now I'm thinking about my own journey.

Having been raised Christian... and in a very evangelical, pentecostal tradition no less... I wasn't exactly encouraged to think like this early on... great swathes of things were simply to be taken on faith, and doubt simply referred over to the appropriately doctrinal section of apologetics. It did not help that both of my parents hold bachelor's degrees in Christian theology.

Nonetheless... they couldn't entirely shield me from the world. The start of my journey, so far as I can recall, came while studying physics... which I did before high school, incidentally, as I was always a voracious reader. I read about things like Einstein's Theory of Relativity, and Heisenberg's Uncertainty Principle... and after thinking about the implications of them, I realized that I was confused.

"Hang on, this can't be right. God is omniscient, and thus has knowledge of everything, everywhere in the universe, irregardless of such pettiness as the speed of light being the maximum speed for transfer of information. The tools of Science may not be able to determine both the exact position and momentum of a particle, but God must know!"

Of course, my initial reaction was wrong. But I'd spotted a problem in my worldview, and additional study of science seemed to reinforce my nagging doubts. I didn't have a word to explain it, then... indeed, I didn't have the words to fully articulate the problem until I read Atlas Shrugged in college, and thereby learned of the law of non-contradiction. But it was seeing that contradiction that first got me thinking in terms of "either my religious beliefs are true, or a large body of experimentally verified science is true, but not both"... and having realized that, it left me open to changing my mind on the subject later on. Which, fortunately, I did.

The first little heuristic trick toward mental health. It seems to me that children actually used to be taught such things, and weren't left to bring themselves up like wolf boys.

For some reason, this article had my mind wandering to "My Favorite Things". Turns out, there was a reason. Think Julie Andrews, and sing along:

When the dog bites, when the bee stings When I'm feeling sad, I simply remember my favorite things and then I don't feeel so bad!

Weren't children's stories full of such things, once upon a time? Every story had some lesson to be learned. I don't have kids, but my impression is that every story these days aims at imparting a proper attitude, not a useful skill.

I have a similar story.

When I was a senior in highschool, I took microeconomics. Before senior year I never tried in school. Senior year I started to care about intellectual things and started trying. I hated my teacher and openly read the textbook in class instead of listening to her lectures.

When it came time for finals, I had a good grade in the class, some sort of high A, and had already been accepted to college. I figured out that if I don't do the final project, my grade would be a B.

I told this to my teacher. She was so angry at me and said she'd fail me. I said she can't, at least not without going against the grading rubric she had established in our syllabus.

I explained to her that I was frustrated because she was missing the most important concept in all of economics: incentives! How can she design the incentives and then be mad at me for acting accordingly? What kind of economist does that? Moreover, the whole system is set up with bad incentives. She should thank me for exposing them, recognize that I've grasped the important concepts, and give me an A.

At least that's the story I like to tell myself. And other people! It's not the truth though. The truth is I have major public speaking anxiety and didn't want to do the public speaking the final project would entail. That's all it was. I don't even think I grasped the centrality of incentives at the time. I think I learned about that about a year later once I discovered LessWrong.

Well, even that paragraph isn't really the truth, but it's something I tell people sometimes too. The reason I tell people these untrue stories is when I figure the truth doesn't matter much and because the story is just meant to convey a point, not be a description of who I am as a person. This comment itself is probably being made less clear and more awkward by this last paragraph, and I considered not including it for that reason. But things like that are a bit of a slippery slope. I noticed myself starting to believe the fake stories to some extent, and that's probably not worth the price of telling a less awkward story to people who I don't care about.