LessWrong 2.0 Reader

It’s not just the title but also (what I took to be) the thesis statement “Comparisons are often useful, but in this case, I think the disanalogies are much more compelling than the analogies” and “While I think the disanalogies are compelling…”

For comparison:

Question: Are the analogies between me (Steve) and you (David) more compelling, or less compelling, than the disanalogies between me and you?

I think the correct answer to that question is “Huh? What are are you talking about?” For example:

• If this is a conversation about gross anatomy or cardiovascular physiology, then you and I are generally extremely similar.
• If this is a conversation about movie preferences or exercise routines, then I bet you and I are pretty different.

So at the end of the day, that question above is just meaningless, right? We shouldn’t be answering it at all.

I don’t think disanalogies between A and B can be “compelling” or “uncompelling”, any more than mathematical operations can be moist or dry. I think a disanalogy between A and B may invalidate a certain point that someone is trying to make by analogizing A to B, or it may not invalidate it, but that depends on what the point is.

FWIW I do think "Biorisk is Often an Unhelpful Analogy for AI Risk," or "Biorisk is Misleading as a General Analogy for AI Risk" are both improvements :)

These are my favorite kinds of posts. Subject expert gives full explanation of optimal resources and methods they used to get where they are.

Curious to hear your thoughts on my sequence: https://www.lesswrong.com/s/raEgRsRo8xWdZoBzh. [? · GW]

I'm going through a theoretical redesign of American public education, and I'd appreciate the feedback from a teacher's perspective.

As it turns out, von Neumann was good at lots of things.

https://qualiacomputing.com/2018/06/21/john-von-neumann/

Von Neumann himself was perpetually interested in many fields unrelated to science. Several years ago his wife gave him a 21-volume Cambridge History set, and she is sure he memorized every name and fact in the books. “He is a major expert on all the royal family trees in Europe,” a friend said once. “He can tell you who fell in love with whom, and why, what obscure cousin this or that czar married, how many illegitimate children he had and so on.” One night during the Princeton days a world-famous expert on Byzantine history came to the Von Neumann house for a party. “Johnny and the professor got into a corner and began discussing some obscure facet,” recalls a friend who was there. “Then an argument arose over a date. Johnny insisted it was this, the professor that. So Johnny said, ‘Let’s get the book.’ They looked it up and Johnny was right. A few weeks later the professor was invited to the Von Neumann house again. He called Mrs. von Neumann and said jokingly, ‘I’ll come if Johnny promises not to discuss Byzantine history. Everybody thinks I am the world’s greatest expert in it and I want them to keep on thinking that.'”

____

According to the same article, he was not such a great driver.

Now, comparing him to another famous figure of his age, Menachem Mendel Schneerson.  Schneerson was legendary for his ability to recall obscure sections of Torah verbatim, and his insightful reasoning (I am speaking lightly here, his impact was incredible). Using the hypothetical that von Neumann and Schneerson had a similar gift (their ability with the written word as a reflection of their general ability), depending on your worldview, Schneerson's talents were not properly put to use in the service of science, or von Neumann's talents were wasted in not becoming a gaon. 

Perhaps, if von Neumann had engaged in Torah instead of science, we could have been spared nuclear weapons and perhaps AI for some time.  Sure, maybe someone else would have done what he did...but who?

I agree that in the limit of an extremely structured optimizer, it will work in practice, and it will wind up following strategies that you can guess to some extent a priori.

I also agree that in the limit of an extremely unstructured optimizer, it will not work in practice, but if it did, it will find out-of-the-box strategies that are difficult to guess a priori.

But I disagree that there’s no possible RL system in between those extremes where you can have it both ways.

On the contrary, I think it’s possible to design an optimizer which is structured enough to work well in practice, while simultaneously being unstructured enough that it will find out-of-the-box solutions very different from anything the programmers were imagining.

Examples include:

• MuZero: you can’t predict a priori what chess strategies a trained MuZero will wind up using by looking at the source code. The best you can do is say “MuZero is likely to use strategies that lead to its winning the game”.
• “A civilization of humans” is another good example: I don’t think you can look at the human brain neural architecture and loss functions etc., and figure out a priori that a civilization of humans will wind up inventing nuclear weapons. Right?

The problem with too-hungry people is that yes, they eat too much, and no, they cannot just stop. The way it works is so: they eat a meal, start feeling very hungry an hour later, make an effort and resist eating for another hour, then some more until they can't anymore and they give in and gobble up everything in the fridge. In cases like this skipping dinner is not actually possible, at least long-term.

I would consider adding salt to something to be making that thing less healthy. If adding salt is essential to making something edible, I think it would be healthier to opt for something that doesn't require added salt. That's speaking generally though, someone might not be getting enough sodium, but typically there is adequate sodium in a diet of whole foods.

I'd say it's too strong a claim that adding salt makes things less healthy. Remember that humans, eating generally mostly plants but some meat as well, developed rather strong craving for sodium salt, just like most herbivorous mammals. If you eat enough meat (not boiled) you don't need more sodium, if you eat a little meat or a lot but boiled, it's probably better to add some. If you eat only potatoes, you'll die without added salt (just kidding, who eats only potatoes).

In terms of factorizing or fingerprinting, 20 Tarot concepts seems like a lot; it's exhausting even just to skim it. Why do you think you need so many and that they aren't just mostly many fewer factors like Big Five or dirty uninterpretable mixes? Like the 500 closest tokens generally look pretty random to me for each one.

If this is the main reason why we should avoid ultra-processed food, then of course we'll have to avoid seed oils at any cost, as those are both ultra-processed and rather new, certainly not what we were evolved to breathe in at sea level.

I wasn't able to find the full video on the site you linked, but I found it here, if anyone else has the same issue: