LessWrong 2.0 Reader

Huh, cool. Intuitively, I'd expect those character-level similarities not to matter too much since the tokenization makes these end up in very different parts of embedding space, unless "kwiecień" or "kviten" are often misspelled as words with the prefix "kwiet". (I check with Google translate, which ~always translates "kwiet" as "quiet" for Slavic languages & Maltese, and as "flower" in Polish).

I'm surprised to see so little discussion of educational attainment and it's relation to birth order here. It seems that a lot of the discussion is around biological differences. Did I miss something?

Families may only have enough money to send one child to school or university, and this is commonly the first born. As a result, I'd expect to see a trend of more first-borns in academic fields like mathematics, as well as on LessWrong.

As a quick example to back up this hunch, this paper seems to reach the same conclusion:

https://www.sciencedirect.com/science/article/abs/pii/S0272775709001368

"birth order turns out to have a significant negative effect on educational attainment. This decline in years of schooling with birth order turns out to be approximately linear."

I'd be interested if the effect still exists if we control for educational attendance/ resources somehow.

Yeah, definitely not the least likely trajectories, instead it's just the next token with the smallest probability. I was thinking of doing beam search with minimizing logits, but that looked difficult to implement. Still surprised that it produces things like prü|stor|oire| which are pretty pronounceable.

+1 to Taleb's Extremistan vs Mediocristan model

One question I have that might be relatively tractable: if I'm using plastic containers for leftovers, how much difference is there between

1. Store in the container, put on plate to microwave and eat.
2. Store and microwave in the container, put on plate to eat.
3. Who needs a plate anyway? Just eat from the container.

The bit about plastic chopping boards kind of hints that (3) might give a lot more microplastics than (2)? But you're probably less violent to the container than the chopping board.

it will also set off the enemy rhetoric detectors among liberals

I'm not sure about that, does Bernie Sanders rhetoric set off that detector?

Current LLMs are capable of solving novel problems when the user does most the work: when the user lays the groundwork and poses the right question for the LLM to answer.

So, if we can get LLMs to lay the groundwork and pose the right questions then we'll have autonomous scientists in whatever fields LLMs are OK at problem solving.

This seems like something LLMs will learn to do as inference-time compute is scaled up. Reasoners benefit from coming up with sub-problems whose solutions can be built atop of to solve the problem posed by the user. 

LLMs will learn that in order to solve difficult questions, they must pose and solve novel sub-questions. 

So, once given an interesting research problem, the LLM will hum away for days doing good, often-novel work. 

In no particular order, because interestingness is multi-dimensional and they are probably all to some degree on my personal interesting Pareto frontier:

• We're not as 3-dimensional as we think [LW · GW]
• Replacing binary questions with "under which circumstances" [LW · GW]
• Almost everything is causally linked [LW · GW], saying "A has no effect on B" is almost always wrong (unless you very deliberately search for A and B that fundamentally cannot be causally linked). If you ran a study with a bazillion subjects for long enough, practically anything you can measure would reach statistical significance
• Many disagreements are just disagreements about labels ("LLMs are not truly intelligent", "Free will does not exist") and can be easily resolved / worked around once you realize this (see also [LW · GW])
• Selection biases of all kind
• Intentionality bias [LW · GW], it's easy to explain human behavior with supposed intentions, but there is much more randomness and ignorance everywhere than we think
• Extrapolations tend to work locally, but extrapolating further into the future very often gets things wrong; kind of obvious, applies to e.g. resource shortages ("we'll run out of X and then there won't be any X anymore!"), but also Covid (I kind of assumed Covid cases would just exponentially climb until everything went to shit, and forgot to take into account that people would get afraid and change their behavior on a societal scale, at least somewhat, and politicians would eventually do things, even if later than I would), and somewhat AI (we likely won't just "suddenly" end up with a flawless superintelligence)
• "If only I had more time/money/whatever" style thinking is often misguided, as often when people say/think this, the sentence continues with "then I could spend that time/money/whatever in other/more ways than currently", meaning as soon as you get more of X, you would immediately want to spend it, so you'll never sustainably end up in a state of "more X". So better get used to X being limited and having to make trade-offs and decisions on how to use that limited resource rather than daydreaming about a hypothetical world of "more X". (This does not mean you shouldn't think about ways to increase X, but you should probably distance yourself from thinking about a world in which X is not limited)
• Taleb's Extremistan vs Mediocristan model
• +1 to Minimalism that lsusr already mentioned
• The mindblowing weirdness [LW(p) · GW(p)] of very high-dimensional spaces
• Life is basically an ongoing coordination problem between your past/present/future selves
• The realization that we're not smart enough to be true consequentialists, i.e. consequentialism is somewhat self-defeating
• The teleportation paradox, and thinking about a future world in which a) teleportation is just a necessity to be successful in society (and/or there is just social pressure, e.g. all your friends do it and you get excluded from doing cool things if you don't join in) and b) anyone having teleported before having convincing memories of having gone through teleportation and coming out on the other side. In such a world, anyone with worries about teleportation would basically be screwed. Not sure if I should believe in any kind of continuity of consciousness, but that certainly feels like a thing. So I'd probably prefer not to be forced to give that up just because the societal trajectory happens to lead through ubiquitous teleportation.

USA is the world government from a money perspective. They can simply tax the world by printing dollars and sending them overseas.

Any lesson learned about decifits/surpluses from the US is suspect.

China's Belt and Road Initiative /New Silk Road means owning parts of other countries is a terminal value.

Other countries mostly have net neutral imports/exports if I remember correctly. 

The way you get rich in an economy is by producing more valuable things and trading for what you want and storing surplus currency (dollars at the world stage).

At times you need to give away your labor so as to start participating and get access, but you need to be getting stuff back to get richer in real terms.

No different than individuals in standard economies

Anyone have a logical solution to exactly why we should act altruistically?

"Logical ... should" sounds like a type error, setting things up for a contradiction. While there are adherents of moral naturalism, I doubt there are many moral naturalists around here. Even given moral naturalism, I believe it would still be true that any amount of intelligence can coexist with any goals [? · GW]. So no, there is no reason why unconstrained intelligences should be altruistic, or even be the sort of thing that "altruism" could meaningfully be asserted or denied of them.

I know it makes sense evolutionarily through game theory and statistics, but human decision making is still controlled by emotions

...which came about through evolution, so what work is the "but" doing? The urge to do good for others is what the game theory feels like from inside [LW · GW].

it's still most advantageous for an individual actor to follow their own self-interest to a degree in a social community.

Each knows their own needs and desires better than anyone else, so it's primarily up to each person to ensure their own are fulfilled. Ensuring this often involves working with others. We do things for each other that we may individually prosper.

So, what type of altruism are you asking about? I expect Peter Singer would dismiss reciprocal altruism as weak sauce, a pale and perverted imitation of what he preaches. The EA variety inspired by Singer? Utilitarianism that values all equally to oneself, and feels another's pain as intensely as one's own? Saintliness that values everyone else above oneself who am nothing? There's a long spectrum there, and people inhabiting all parts of it.