10 Years of LessWrong

post by SebastianG (JohnBuridan) · 2022-12-30T17:15:17.498Z · LW · GW · 2 comments

Contents

2 comments

[I appreciate that less wrong has a very strong norm against navel gazing. Let's keep it that way. The purpose of this post is merely to reflect upon and highlight valuable tools and mental habits in our toolkit.]

The rationalsphere propelled me to where I am today by giving me tools that I wouldn’t otherwise have developed. The concept handles that we have developed are numerous and useful, but here I am only talking about the underlying habits that dug up good ore for the wordsmiths.

It was a typical path starting in 2012. Harry Potter and the Methods of Rationality led to LessWrong, led to Slate Star Codex. Combined they fed into game theory, from game theory economics, and from economics all worked towards a new way of thinking. On a different branch I went from Slate Star Codex to Tetlock and forecasting and participating in some studies and tournaments (I can’t beat the market). On a different branch, I have followed the AI literature from mostly MIRI and Paul Christiano and John Wentworth. And along with the AI DLC in my mind has come “AI development watching” and AI learning systems. This has been great, and it has been standard, and I am sure many others have done the same.

But it was also an atypical path. I was a classicist and philosopher, bouncing around on the back of a bus in Italy between archeological sites with my 17 inch laptop and 12 tabs of HPMOR and two tabs of LW reading on each journey. Not yet having any knowledge of either calculus or discrete math, nor any inkling of basic coding or incentive structures, I was a youngling in the art of rationality. But I was well on my way in the great books tradition.

I read HPMOR and the Sequences angrily. Many of the ideas I found refreshing and so on target, and many more I found blasted, awful, un-nuanced, and wrong. Many ideas about human nature and philosophy of language, logic, and science I wrestled with time and time again - always coming back for more. (Some are still wrong, mind you).

I had rejected psychology sophomore year of college on the grounds that several studies in our textbook obviously didn’t show what they claimed to show, and with that rejection of psychology, I rejected the idea of the quantification of human behavior. But reverse stupidity is not intelligence. So it took about three years before I could be salvaged from that position. It took scores of late night arguments about the foundations of language, logic, math, and science. Those arguments were my gateway into the enterprise, and the LessWrong corpus fueled the fire of those discussions.

LessWrong, from the Sequences, to the community content, to the broader rationalsphere has introduced me to tools and instilled in me habits that I otherwise would not have acquired. To those habits I attribute some of the extraordinary success I have had this past decade. Since I have the unique position of a humanities person coming into the sphere and falling in love with it, I think I have a valuable perspective on what mechanistic, psychological and quantitative tools are the highest leverage for a person initially hostile to the project. Or another way to see it, is that while a child might be initially predisposed to certain habits of thought or pick up those cues from their culture, an outsider-turned-insider might have unique insight about which tools are most salvific for the average person. So I am going to outline in order which concentrated tools that, if turned into habits, significantly elevate one’s sanity.

These will be in the order in which I think they should be taught, not order of importance or “foundationalness.”

1. Think in terms of probabilities. It is hard to imagine a time when this wasn’t obvious. But probabilistic thinking requires humility and attentiveness. A lot of good things follow from developing this habit.
2. Fermi estimate. I think of Fermi estimates as the cousin of thinking in probabilities because it inculcates a willingness to put numbers on things, to think in a bit more detail, and it is in many ways less sophisticated than probabilities and so many people can start doing it.
3. Notice empirical claims. Once you start using Fermi estimates and probabilities you easily notice which claims are empirical and which are not. When you mention to someone that a claim they are making is knowable with data, suddenly the amount of hyperbole available in the conversation starts to get vacuumed out.
4. Shoot for a gears level understanding of things. Once the basic habits have been acquired it becomes much more attractive for the practitioner to shoot for a gears level understanding of many different things which are relevant to them. In that way, one’s error bars in all predictions go down. You start to get a grip on the beasts you are trying to wrestle to the ground.
5. Identify incentives and incidence of costs. This is kind of embarrassing, but I am the type of person who needed to be told that one can and should identify incentive structures, and that people respond to those. I didn’t quite realize incentives existed. I thought there were forces in the world that wanted me to behave one way, and my job was to resist that and pursue a system of values outside of the incentive structures around me. (I still kind of believe this - framing the game is crucial! - but the Prisoner’s Dilemma was a big wake up call to read a lot of Game Theory textbooks). As for the incidence of costs, I am experiencing that right now as a distant relative is making life for others very inconvenient because someone who was sick had contact with someone who had contact with someone who is to come to the New Years’ party. This person has not internalized the costs of behavior on others.
6. (P*V) - C = EV. (Probability times Value) - Cost = Expected value. Do this calculation explicitly and often in management. It is very helpful at giving a rough estimate at what activities are not worth doing. 
7. Bayesian updating. This comes last for a couple reasons. It is difficult. But most importantly it requires the practice and attentiveness of the previous skills.

There are many tools I am leaving out here: linear regression, elasticity, causal inference, computer science concepts, certain biases, and de-biasing tools. A lot of concept handles guide thought in a more productive direction or create quick and useful categorization. Certain useful software that I now implement frequently. Communities and subcommunities where certain skills and practices are honed. There are certain academic fields that I have found very important to be introduced to, but I am not confident this is universally necessary. I am also leaving out all the epistemic changes that have happened to me over the years - too many to count and too boring to recount. Not that many weren’t major. They were huge. But I find it boring to write about right now.

I have never been one to make a hard distinction between instrumental and epistemic rationality. Knowing any true statement might become instrumental years down the line. Like Hamming in The Art of Doing Science and Engineering, I don't see a way for us to clearly distinguish between science and engineering anymore - each person on the cutting edge must be their own scientist and their own engineer, a generalist of sorts. And so, I offer that the proper tools for thought are not an immense oracular processing system, but a mobile mindset, which an individual or a small team can wield in their specific environment.

Developing and refining that mobile system within myself has been the work of this last decade.

I cannot imagine making as much intellectual progress in the next decade as I did the previous one. But I can imagine making far more intellectual contributions, a flowering, and an efflorescence. May it be!

Go team.

 

2 comments

Comments sorted by top scores.