What I've learned from Less Wrong

LW has helped me a lot. Not in matters of finding the truth; you can be a good researcher without reading LW, as the whole history of science shows. (More disturbingly, you can be a good researcher of QM stuff, read LW, disagree with Eliezer about MWI, have a good chance of being wrong, and not be crippled by that in the least! Huh? Wasn't it supposed to be all-important to have the right betting odds?) No; for me LW is mostly useful for noticing bullshit and cutting it away from my thoughts. When LW says someone's wrong, we may or may not be right; but when LW says someone's saying bullshit, we're probably right.

I believe that Eliezer has succeeded in creating, and communicating through the Sequences, a valuable technique for seeing through words to their meanings and trying to think correctly about those instead. When you do that, you inevitably notice how much of what you considered to be "meanings" is actually yay/boo reactions, or cached conclusions, or just fine mist that dissolves when you look at it closely. Normal folks think that the question about a tree falling in the forest is kinda useless; nerdy folks suppress their flinch reaction and get confused instead; extra nerdy folks know exactly why the question is useless. Normal folks don't let politics overtake their mind; concerned folks get into huge flamewars; but we know exactly why this is counterproductive. I liked reading Moldbug before LW. Now I find him... occasionally entertaining, I guess?

Better people than I are already turning this into a sort of martial art. Look at Yvain cutting down ten guys with one swoop, and then try to tell me LW isn't useful!

Going back and looking at the sequences is funny. Across many posts, comments accuse Eliezer of simplifying and attacking straw-men. But as someone who was religious when he was first reading OB, and who got deconverted specifically because of the arguments therein, I think that Eliezer had it right and the accusers had it wrong: many of the arguments he refutes seem like straw-men to people who associate with other rationalists, but to those steeped in irrationality they are basically the world. Witness, for instance, a former Christian's revisiting of CS Lewis to find that he not only fails to provide a strong defense of Christianity, he's basically a joke to anyone who knows enough history or biology or sociology or psychology. But when you're in an affective death spiral, you often can't notice such things.

While I worry about the self-congratulation of threads like these, I want to nominate a lesson I learned from Robin Hanson (and Daniel Klein, another GMU economist), which will probably affect me professionally as much as my religious deconversion affected me personally:

It is ok to believe things that are obvious, even if they are unpopular.

It seems non-controversial, but when you actually find yourself in a discussion with an intelligent, like-minded person with similar interests and arguments with the backing of high-status individuals, the temptation is enormous to switch sides.

See, this is the way to get people to read the posts in the sequences: give them a reason and speak personally. For example, you've just given me another attack of tab explosion ...

My gains from LessWrong have come in several roughly distinct steps, all of which have come as I've been working my way through the Sequences. (Taking notes has really helped me digest and cement the information.)

1) Internalizing that there is a real world out there, and like Louie said, that ideas can be right or wrong. Making beliefs pay rent, referents and references, etc. A perspective on beliefs that they should accurately reflect the world and be used for achieving things I care about; all else is fluff. That every correct map should agree with every other, so that life does not seem such a disconnected jumble of different domains. Overall these kinds of insights really helped to give focus to my thoughts and clear out the clutter of my mind.

2) Having a conception of what beliefs should do, LessWrong helps me be aware of and combat various biases that interfere with the formation of accurate beliefs, and with taking coherent action based on those beliefs. I've made large gains here, but of course I'm not finished.

3) Forming a coherent, productive, happy me. Bootstrapping and snowballing effects. As I learn more, I seek out more good information, better. On this point, see Anna Salamon's posts going back to "Humans are not automatically strategic." The book "The Art Of Learning" by Josh Waitzkin has been immensely helpful. Learning about Cognitive Behavioral Therapy (this book is good) has been very helpful in being empirical and rational about the self. I believe this is basically the material of the Luminosity sequence, though I read those posts some time ago and should probably review them.

There's far too much to go into specifically, but the transformation has been huge, and continues. When conversing with non-rationalists, arguments feel like a match between Bruce Lee and some guy off the street. It's not that I have any more raw intellectual power than I had before, but my set of tools/training has improved tremendously. Unfortunately without being a rationalist, a non-rationalist doesn't (much) realize the extent to which they're outmatched, and indeed there is seldom a point to "beating someone." Instead you realize that even a poorly argued-position can be correct, looking out for points you may have missed, and perhaps try and introduce a few concepts. It feels like I'm working at a level above most people, that the conversation is a different thing to me and them; it's not like I can just tell them all this. I discovered LessWrong through an interest in existential risk and at first it seemed kind of boring, not very useful, this weird academic exercise. I wish I could convey to more people how helpful it's been, and the extent to which I didn't know what I didn't know.

(A note on the community: I think it's great that it's here, and I think that some really great material has been produced, and continues to be produced, beyond the "core" material by Eliezer. That said, I almost never read comments and I only read front-page, promoted posts; the return on time for reading anything else doesn't seem great enough right now, compared to my other work and studies. Just to give an idea on how I'm using LessWrong.)

I've been looking around the site for awhile, having several people I know who go here. What I've learned is unfortunately that I'm unlikely to be able to learn from this site unless something changes. Which is too bad because I don't think I'm unable to learn in general.

I have no academic background whatsoever, and no expertise in science or philosophy. I am not an intellectual. I am good at noticing jargon, but terrible at picking it up and being able to use and understand it. I have no particular skill in abstract thinking. While tests aren't everything, I score in the range of borderline intellectual functioning on IQ tests and I do so for a reason: I am quite lacking in several standard cognitive abilities.

I also have obvious cognitive strengths, writing among them, but they don't match up with the ones necessary to navigate this site. From my perspective, reading this site is like trying to read a book with several words per sentence chopped out, and the words that remain being used in /ways/ that don't match well with my ability to comprehend.

Normally I would just turn around and walk away. I don't think anyone here has any particular desire to see someone like me shut out. I find it saddening though that a site dedicated to helping people think more accurately is mostly dominated by people who have a good deal of intellectual skills already. I would be curious to see how the ideas here could be modified to assist people who are not typical users here. People who can't read mountains of text in order to prepare themselves for the conversations that are taking place, and who need things explained in ways that are understandable even if you're average or even a slow learner.

This isn't meant as an attack, just a suggestion for new directions the site could take in order to benefit people who aren't all that intellectual. You don't have to have all the traditional cognitive abilities to appreciate the importance of thinking clearly about reality. I even bet that the techniques would have to be modified for some of us who can't hold complex ideas in our heads. But modifying them would be a good way to show there's more than a single set of cognitive techniques to get to the same goal of understanding the world as accurately as possible.

Very nice post! My personal favorite things I've learned about from reading LessWrong:

• Causality: Models, Reasoning, and Inference, a book by Judea Pearl written in 2000 which is frequently referenced by the SIAI and on LessWrong.

• Spaced Repetition Software.

• Politics as charity: that in terms of expected value, altruism is a reasonable motivator for voting (as opposed to common motivators like "wanting to be heard").

• That a significant number of people are productively working on philosophical problems relevant to our lives.

• Lots of little sanity checks to keep in mind, like Conservation of Expected Evidence, i.e. that without evidence, your expectation of what your confidence will be after seeing evidence is equal to your prior confidence. (But see this comment on things you can expect from your beliefs.)

I can't claim to be "converted to rationality" or any particular school of thought by LessWrong, because most of the ideas in the sequences were not new to me when I read them, but it was extremely impressive and relieving to see them all written down in one place, and they would have made a huge impact on me if I'd read them growing up!

And many of the people in this community rub me the wrong way.

Yes, like you, for stealing my post idea! Kidding, obviously.

At the risk of contributing to this community becoming a bit too self-congratulatory, here are some of the more significant concepts that I've grokked from reading LW:

• No Universally Compelling Arguments and Ghosts in the Machine. Shamefully, it never even occurred to me to de-anthropomorphize the idea of a mind.

• You Provably Can't Trust Yourself and No License To Be Human, along that same theme.

• The Luminosity sequence is a bit under-celebrated, I think, with relation to its value. I've found that to be one of the most important things I've read here, and applying those concepts has aided me in improving my life in not-insignificant ways.

• Affective Death Spirals! I cannot praise this enough for giving me the skills to recognize this phenomenon and keep myself from engaging in this at the negative end.

Most of all, LW has taught me that being the person that I want to be takes work. To actually effect any amount of change in the world requires understanding the way it really is, whether you're doing science or trying to understand your own personality flaws. Refusing to recognize said flaws doesn't make them go away, reality doesn't care about your ego, etc.

And apparently there was this Bayes guy who had a pretty useful theorem...

Interestingly, although reading the Sequences and other LW articles significantly affected my thinking style and general outlook over time, I've probably learned as much if not more from participating -- writing posts and comments, and receiving feedback.

...which feels strange to say, because I was skeptical in the beginning of the whole transition of Overcoming Bias into LW. For one thing, I didn't like the idea of having to "move". And I was highly suspicious of the karma system, because I was afraid of having my status numerically measured. I had been perfectly content to sit back and passively read Hanson and Yudkowsky posts, skim the comments, and only rarely chime in with a comment of my own when I thought it was particularly important.

But now, I think the interactive, community aspect of LW is probably its greatest feature.

Great post!

My experience on Less Wrong has been that many of the top-voted articles initially have seemed sort of mundane and obvious if mildly pleasant to read, but that returning to them and having them reverberate in my mind has been very helpful to me in framing the issues that come up in my day to day life. Over and over again I've had the experience of being subliminally aware of a given phenomenon discussed on Less Wrong but that reading a well-written explanation is very helpful to me in drawing the key issues at hand into focus.

1. Eliezer's articles listed under Shut Up and Multiply helped me become more comfortable with expected utility theory. (Disclaimer: I do not fully agree with all points that he makes therein.)

2. Yvain's The Trouble With Good and Missing the Trees for the Forest have been helpful to me in dispelling halo effects.

3. I'm continually amazed by how relevant I find Yvain's Generalizing From One Example and Typical Mind and Politics to my own life and to understanding the thinking of others.

4. I'm a very unusual person and have had little opportunity to meet people who I have a lot in common with in the past. On Less Wrong I've found some people who think in terms similar to the ones that I do and interacting with them has given me the opportunity to trade insights with them, about self-improvement, about the world at large and about interacting with more mainstream people.

5. I think that people's willingness to engage with those who disagree with them is noticeably higher (on average) on Less Wrong than it is on most online forums. A major benefit that I've reaped from this is that I've learned more about communication with those who have different worldviews here than I would have had the chance to elsewhere. I describe a special case of this in the conclusion to my Reflections on a Personal Public Relations Failure: A Lesson in Communication posting.

I've learned that people significantly more knowledgeable and intelligent than me do exist, and not just as some mythical statistical entity at the fringes of what I'll realistically encounter in my everyday life.

The internet - and indeed communications technology in general - is beneficial like that, even if it takes some searching to find a suitable domain.

I have learned that philosophy remains a big unsolved problem where no one seems to have really gotten anywhere for a long time, yet concerted effort by determined smart people might lead to us answering some of the most important questions that have always plagued human philosophers. I have learned that solving philosophy (where philosophy includes questions like "what is human value?", "what is the nature of intelligence?", "what are the simple equations that unify the physical laws of our universe/multiverse?") is of importance on a mind bogglingly cosmological level.

Keep on thinking, friends.

Even when most peoples beliefs are junk you won't know before you considered the belief in detail. You probably just increase the effect of confirmation bias when you reject beliefs without examining them.

Thinking outside your own set of beliefs is also good training.

I learned that humans are all very alike.

I learned that natural selection uses up diversity.

I learned some more graceful words and arguments for what I wanted than I had. For instance, previously I explained that I think about religion logically because I used to be Catholic and we do that, now I can say that it is because logic is useful for thinking about everything and tell people my backstory later if they ask.

I learned that emotion and rationality are not enemies. Vulcans we are not.

I learned that normally rational people will take sides in emotional name calling once you blame them. Much like everyone else. (See most any mention of gender.)

That it is possible to take confusing issues and write clearly about them.

That this may require sequences.

I've found out about PJ Ebys ideas and even though I just recently managed to use them to make a substantial change, I'm pretty sure it's the largest positive change in my entire life so far.

I learned that meditation can be fun, and there are instructions available.

I learned that trying to get an exact definition of a term can be futile, since the meaning in one's mind is structured more like a simple artificial neural network than like the expected kind of verbal definition. Examples: "what is science fiction", "what is a fish".

It states that Less Wrong is a blog devoted to refining the art of rationality. Rationality is about winning and you and me and the rest of humanity can only win if we are able to solve the problem of provably Friendly AI. What I have learnt is that one should take risks from artificial intelligence serious. And I still believe that it is the most important message Less Wrong is able to convey.

Why shouldn't the discussion of risks posed by AI be a central part of this community? If risks from artificial intelligence are the most dangerous existential risk that we face how is it not rational to inquire about it and try to improve how this risk is communicated towards outsiders?

“The primary thing when you take a sword in your hands is your intention to cut the enemy, whatever the means. Whenever you parry, hit, spring, strike or touch the enemy’s cutting sword, you must cut the enemy in the same movement. It is essential to attain this. If you think only of hitting, springing, striking or touching the enemy, you will not be able actually to cut him. More than anything, you must be thinking of carrying your movement through to cutting him.” via Twelve Virtues of Rationality

This is what I learnt and that the enemy is unfriendly AI.

"Most peoples' beliefs aren’t worth considering ... dropping the habit of seriously considering all others’ improper beliefs that don’t tell me what to anticipate and are only there for sounding interesting or smart."

Seems you assume that most peoples' beliefs are "improper." Did LW offer you evidence for that conclusion? And don't you also need to assume you have a way to generate beliefs that is substantially better at avoiding the desire to sound interesting or smart?

The most important thing I learned from LessWrong is that my brain isn't always right.

This was a huge thing for me.

I already had the reductionist viewpoint, that I was just a brain. But I only had a part of it. I basically presumed that my thought processes were right: they couldn't be wrong, since if they were wrong, correcting it was merely a matter of changing some of the biological structures and firing patterns. But since I was that structure and those patterns, the 'corrected' version wouldn't be me; it would be someone else. The way I was, was the only way I could be, if I wanted to be me. So I had what you might call biological relativism.

The sequences' focus on biases let me realise that wasn't the case.

Not only is the free will problem solved, but it turns out it was easy.

Haha. Ha.

Although it is easy to resolve it to your own satisfaction, it is more difficult to resolve it to other peoples' satisfaction. Which suggests that there is a problem, at least if you want to avoid retreating to fully general counterarguments like "you disagree with me, so you must be irrational." A quote comes to mind here: "The first principle is that you must not fool yourself - and you are the easiest person to fool." - Richard Feynman.

A good resource for getting you to ask the right questions about free will might be Yvain's excellent post.

In general, though, thank you for summarizing what you've learned, it was interesting.

Right, I've read the solution sequence to "free will" and all I've managed to glean from it is that a) I'm physics, whose ontology I'm quite ignorant of and b) free will is conceptually incoherent and needs dissolving. I certainly don't feel like or believe I have free will or that I could influence the creation of FAI by desire for example. Is there something Louis(me) is missing that Louie isn't from the sequence? I find the sequence too long and prosaic to fit in my head to make a visceral impact. Is there a more concise alternative or even just an alternative that would make Louis.belief == Louie.belief? I'm struggling guys please help.

I will read more into this strange rationalist blog, but so far the blog does seem rather arrogant to me. Every other post claims to "solve" a traditional philosophical problem "easily". This post doesn't even bother to do that; it just states the problems have been easily solved. What have I seen so far: the problem of induction, the problem of free will, what the correct meta ethics is, an exact analysis of belief, a proof of metaphysical realism. I hope the writers in this blog are aware that countless people throughout history, with a wide variety of viewpoints, have presented supposedly "easy" solutions to these diverse problems. We tend to view the ones further back in history as philosophers, and the ones closer (from the 20th century onwards) as bad philosophers. Not always. There have been some serious modern attempts to easily solve philosophical problems. Nevertheless, this blog does come off as arrogant. For an outsider like myself, I could give a lot more of my confidence to this blog if any of it had been published in an academic journal. I don't know if any has; please direct me if it has.

Number 4 is totally wrong.

"In order to be able to think up a hypothesis which has a significant chance of being correct, I must already possess a sufficient quantity of information" is obvious, following immediately from the mathematics of information. But that's emphatically not the same thing as "I obtain my hypothesis by applying a 'principle of induction' to generalize the data I have so far."

The way induction was supposed to work was that your observation statements served as the premises of a kind of inference. Just as one can use deductive logic to infer "Swan[1] is white" from "All swans are white", so one was supposed to be able to infer "All swans are (probably) white" from ("Swan[1] is white", ..., "Swan[N] is white") for sufficiently large N.

But there is no such thing as a "method of induction" which finds hypotheses for you. Consider those swans: In order to even write down the data we needed to have the concepts 'white' and 'swan', and something must have motivated us to look specifically at swans and note down specifically what colour they are. In other words, by the time we get round to actually applying our "method of induction" we must already have formed the very hypothesis that the method was supposed to return, or something close to it (like "all swans are some colour - perhaps white").

This becomes comical when we turn to GR:

Our raw, unprocessed 'sense data' comes streaming in: ("The precession of Mercury's perihelion looks exactly as if (long description of the mathematics of general relativity)", "The apparent position of this star as the sun moves in front of it changes in a manner that looks exactly as if (long description of the mathematics of general relativity)", "A clock aboard this high-flying plane runs slightly faster than one on earth, exactly as if (long description of general relativity)") ... and then, as if by magic, the Method Of Induction selects for us the appropriate hypothesis: (long description of the mathematics of general relativity).

I enjoyed the post (enough for a vote up!) but I find myself wishing it had stopped at #5.

6 is mostly correct but has significant edge cases (even if you subscribe to MWI, probabilities pop up when dealing with tiny things). Something like "Probabilities exist in minds" is a much more agreeable statement than "Probabilities don't exist elsewhere," and has the same framing benefits.

7 just flat out bothers me. Many Worlds is just an interpretation, a flavor- it shares the exact same math with all other flavors of quantum mechanics. I agree with Eliezer that it's a far more agreeable flavor than Copenhagen- but those aren't the only two flavors available. And if you are making predictions based on your flavor preferences, something went wrong somewhere. I cannot see how your tastes when it comes to QM should impact whether or not you sign up for cryonics with the currently existing firms offering cryonic services.

How constructive is:

1. Beliefs are for controlling anticipation (Not for being interesting)

...? ...since beliefs, do in fact, serve all kinds of signalling purposes among humans.