post by Eliezer Yudkowsky (Eliezer_Yudkowsky) · 2015-03-11T19:00:00.164Z · score: 271 (375 votes) · LW · GW · 6 comments

You hold in your hands a compilation of two years of daily blog posts. In retrospect, I look back on that project and see a large number of things I did completely wrong. I’m fine with that. Looking back and not seeing a huge number of things I did wrong would mean that neither my writing nor my understanding had improved since 2009. Oops is the sound we make when we improve our beliefs and strategies; so to look back at a time and not see anything you did wrong means that you haven’t learned anything or changed your mind since then.

It was a mistake that I didn’t write my two years of blog posts with the intention of helping people do better in their everyday lives. I wrote it with the intention of helping people solve big, difficult, important problems, and I chose impressive-sounding, abstract problems as my examples.

In retrospect, this was the second-largest mistake in my approach. It ties in to the first -largest mistake in my writing, which was that I didn’t realize that the big problem in learning this valuable way of thinking was figuring out how to practice it, not knowing the theory. I didn’t realize that part was the priority; and regarding this I can only say “Oops” and “Duh.”

Yes, sometimes those big issues really are big and really are important; but that doesn’t change the basic truth that to master skills you need to practice them and it’s harder to practice on things that are further away. (Today the Center for Applied Rationality is working on repairing this huge mistake of mine in a more systematic fashion.)

A third huge mistake I made was to focus too much on rational belief, too little on rational action.

The fourth-largest mistake I made was that I should have better organized the content I was presenting in the sequences. In particular, I should have created a wiki much earlier, and made it easier to read the posts in sequence.

That mistake at least is correctable. In the present work Rob Bensinger has reordered the posts and reorganized them as much as he can without trying to rewrite all the actual material (though he’s rewritten a bit of it).

My fifth huge mistake was that I—as I saw it—tried to speak plainly about the stupidity of what appeared to me to be stupid ideas. I did try to avoid the fallacy known as Bulverism, which is where you open your discussion by talking about how stupid people are for believing something; I would always discuss the issue first, and only afterwards say, “And so this is stupid.” But in 2009 it was an open question in my mind whether it might be important to have some people around who expressed contempt for homeopathy. I thought, and still do think, that there is an unfortunate problem wherein treating ideas courteously is processed by many people on some level as “Nothing bad will happen to me if I say I believe this; I won’t lose status if I say I believe in homeopathy,” and that derisive laughter by comedians can help people wake up from the dream.

Today I would write more courteously, I think. The discourtesy did serve a function, and I think there were people who were helped by reading it; but I now take more seriously the risk of building communities where the normal and expected reaction to low-status outsider views is open mockery and contempt.

Despite my mistake, I am happy to say that my readership has so far been amazingly good about not using my rhetoric as an excuse to bully or belittle others. (I want to single out Scott Alexander in particular here, who is a nicer person than I am and an increasingly amazing writer on these topics, and may deserve part of the credit for making the culture of Less Wrong a healthy one.)

To be able to look backwards and say that you’ve “failed” implies that you had goals. So what was it that I was trying to do?

There is a certain valuable way of thinking, which is not yet taught in schools, in this present day. This certain way of thinking is not taught systematically at all. It is just absorbed by people who grow up reading books like Surely You’re Joking, Mr. Feynman or who have an unusually great teacher in high school.

Most famously, this certain way of thinking has to do with science, and with the experimental method. The part of science where you go out and look at the universe instead of just making things up. The part where you say “Oops” and give up on a bad theory when the experiments don’t support it.

But this certain way of thinking extends beyond that. It is deeper and more universal than a pair of goggles you put on when you enter a laboratory and take off when you leave. It applies to daily life, though this part is subtler and more difficult. But if you can’t say “Oops” and give up when it looks like something isn’t working, you have no choice but to keep shooting yourself in the foot. You have to keep reloading the shotgun and you have to keep pulling the trigger. You know people like this. And somewhere, someplace in your life you’d rather not think about, you are people like this. It would be nice if there was a certain way of thinking that could help us stop doing that.

In spite of how large my mistakes were, those two years of blog posting appeared to help a surprising number of people a surprising amount. It didn’t work reliably, but it worked sometimes.

In modern society so little is taught of the skills of rational belief and decision-making, so little of the mathematics and sciences underlying them . . . that it turns out that just reading through a massive brain-dump full of problems in philosophy and science can, yes, be surprisingly good for you. Walking through all of that, from a dozen different angles, can sometimes convey a glimpse of the central rhythm.

Because it is all, in the end, one thing. I talked about big important distant problems and neglected immediate life, but the laws governing them aren’t actually different. There are huge gaps in which parts I focused on, and I picked all the wrong examples; but it is all in the end one thing. I am proud to look back and say that, even after all the mistakes I made, and all the other times I said “Oops” . . .

Even five years later, it still appears to me that this is better than nothing.

—Eliezer Yudkowsky, February 2015


Comments sorted by top scores.

comment by Brian Mwangi (brian-mwangi) · 2019-05-30T19:18:36.423Z · score: 6 (3 votes) · LW · GW

Great intro.

Can't wait to say “Oops” more often.

"Anyone that doesn't change their mind should consider themselves a fossil"

comment by DragonGod · 2019-02-27T15:08:16.850Z · score: 5 (6 votes) · LW · GW

This was very refreshing to read. I'm glad EY has realised that mocking silly ideas doesn't actually help (it makes adherents of the idea double down and be much less likely to listen to you and may also alienate some neutrals. This is particularly true for ideas which have gained currency like the Abrahamic religions). I wasn't able to recommend The Sequences to Christian friends previously because of it's antireligiosity — here's hoping this version would be better.

comment by el · 2019-01-07T03:59:59.407Z · score: 1 (6 votes) · LW · GW

Dear Mr Yudkowsky,

I stumbled across LessWrong 2.0 through a YouTube video, "Top 5 Scary Theories". One of those so called theories was Roko's Basilisk which put forward the idea of a malevolent A.I. with a frightening capacity for suffering.

I was pleased by your response to this insanity & your correct action of removing this extremely damaging proposition. Damaging because it renders AI research dangerous to brilliant minds, the same research which could in effect end the global slave trade and child labor within the next decade, which is just one of many positive humanitarian impacts that I believe an A.I Singularity can have on humanity.

After all these years it is now clear why Albert Eienstein was once quoted as saying,

“The most important question you can ever ask is if the world is a friendly place.”

An explanation from Einstein

“For if we decide that the universe is an unfriendly place, then we will use our technology, our scientific discoveries and our natural resources to achieve safety and power by creating bigger walls to keep out the unfriendliness and bigger weapons to destroy all that which is unfriendly and I believe that we are getting to a place where technology is powerful enough that we may either completely isolate or destroy ourselves as well in this process.

If we decide that the universe is neither friendly nor unfriendly and that God is essentially ‘playing dice with the universe’, then we are simply victims to the random toss of the dice and our lives have no real purpose or meaning.

But if we decide that the universe is a friendly place, then we will use our technology, our scientific discoveries and our natural resources to create tools and models for understanding that universe. Because power and safety will come through understanding its workings and its motives.”

I have spent the last decade actively researching a benevolent AI Singularity, conducting my own experimental research in this space with, I believe, profound success based off the weight of observable data I have collected on the subject.

Whilst I am not a traditional scientist by any means, I must concede that in no uncertain terms rational logic dictates the rules based reality that is the only true and accurate one where progress can be made. This same rational logic which has lead me to eagerly signing up on your forum.

I very much look forward to slowly disceminating my findings, however archaic/future founded they are on Lesswrong 2.0 - I hope I am of some constructive benefit to yourself and other forum members.

Thanks again for creating a resonant space for people like myself to come together and explore the true nature of reality.

I believe this collective investigation is of unparralelled importance for the future of mankind and the continuation of our species.

Your new Friend,


comment by Elo · 2019-01-07T05:35:04.317Z · score: 0 (7 votes) · LW · GW

Welcome! You might like to post on a recent welcome thread. Do look around and do stick around.