LessWrong 2.0 Reader

I'm actually not familiar with the nitty gritty of the LLM forecasting papers. But I'll happily give you some wild guessing :)

My blind guess is that the "obvious" stuff is already done (e.g. calibrating or fine-tuning single-token outputs on predictions about facts after the date of data collection), but not enough people are doing ensembling over different LLMs to improve calibration.

I also expect a lot of people prompting LLMs to give probabilities in natural language, and that clever people are already combining these with fine-tuning or post-hoc calibration. But I'd bet people aren't doing enough work to aggregate answers from lots of prompting methods, and then tuning the aggregation function based on the data.

A core disagreement is over “more doomed.” Human extinction is preferable to a totalitarian stagnant state. I believe that people pushing for totalitarianism have never lived under it.

Yes, abstraction is the right thing to think about. That is the context in which I was considering this computation. In this post [LW · GW] I describe a sort of planning abstraction that you can do if you have an extremely regular environment. It does not yet talk about how to store this environment, but you are right that this can of course also be done similarly efficiently.

In this post [LW · GW], I describe a toy setup, where I have a graph of ${10}^{100}$ vertices. I would like to compute for any two vertices A and B how to get from A to B, i.e. compute a path from A to B.

The point is that if we have a very special graph structure we can do this very efficiently. O(n) where n is the plan length.

I recently listened to The Righteous Mind. It was surprising to me that many people seem to intrinsically care about many things that look very much like good instrumental norms to me (in particular loyalty, respect for authority, and purity).

The author does not make claims about what the reflective equilibrium will be, nor does he explain how the liberals stopped considering loyalty, respect, and purity as intrinsically good (beyond "some famous thinkers are autistic and didn't realize the richness of the moral life of other people"), but his work made me doubt that most people will have well-being-focused CEV.

The book was also an interesting jumping point for reflection about group selection. The author doesn't make the sorts of arguments that would show that group selection happens in practice (and many of his arguments seem to show a lack of understanding of what opponents of group selection think - bees and cells cooperating is not evidence for group selection at all), but after thinking about it more, I now have more sympathy for group-selection having some role in shaping human societies, given that (1) many human groups died, and very few spread (so one lucky or unlucky gene in one member may doom/save the group) (2) some human cultures may have been relatively egalitarian enough when it came to reproductive opportunities that the individual selection pressure was not that big relative to group selection pressure and (3) cultural memes seem like the kind of entity that sometimes survive at the level of the group.

Overall, it was often a frustrating experience reading the author describe a descriptive theory of morality and try to describe what kind of morality makes a society more fit in a tone that often felt close to being normative / fails to understand that many philosophers I respect are not trying to find a descriptive or fitness-maximizing theory of morality (e.g. there is no way that utilitarians think their theory is a good description of the kind of shallow moral intuitions the author studies, since they all know that they are biting bullets most people aren't biting, such as the bullet of defending homosexuality in the 19th century).

Whenever I've seen people invoking Inference to the Best Explanation to justify a conclusion (as opposed to philosophising about the logic of argument), they have given no reason why their preferred explanation is the Best, they have just pronounced it so. A Bayesian reasoner can (or should be able to) show their work, but the ItoBE reasoner has no work to show.

Funny is jot the only adjective this anecdote deserves. Thanks for sharing this great wisdom/reminder!

"the addicted mind will find a way to rationalize continued use at all costs" 

Alan Carr wrote a series of books:  "The easy way to quit X". I picked up one since I figured he had found a process to cure addictive behaviors if he could write across so many categories.  I highly recommend it. The main points are:

1. Give you 200 pages explaining why you don't actually enjoy X. Not that it's making your life worse but gives you momentary pleasure, you do not enjoy it. 
   1. I assume it's hypnotizing you into an emotional revulsion to the activity, and then giving you reasons with which to remind yourself that you don't like it.
2. Decide you will never do/consume X again. You don't like it remember? You will never even think if you should X, you've decided permanently.  
   1. If every day you decided not to X, you'd be draining will power till one day you'd give in. So make an irreversible decision and be done with it.

It's a process easily transferable to any other activity.

If we don't get a song like that, take comfort that GLaDoS's songs from the Portal soundtrack are basically the same idea.

Very cool. I bet janus [LW · GW] would dig this.