LessWrong 2.0 Reader

Even assuming that the simulators have wildly different values, why would doing something insane a good thing to do?

(Fun tangent, not directly addressing this argument thread.)

There’s a trio of great posts from 2015 by @JonahS [LW · GW] : The Truth About Mathematical Ability [? · GW] ; Innate Mathematical Ability [? · GW] ; Is Scott Alexander bad at math? [LW · GW] which (among other things) argues that you can be “good at math” along the dimension(s) of noticing patterns very quickly, AND/OR you can be “good at math” along the dimension(s) of an “aesthetic” sense for concepts being right and sensible. (My summary, not his.)

The “aesthetics” is sorta a loss function that provides a guidestar for developing good deep novel understanding—but that process may take a very long time. He offers Scott Alexander, and himself, and Alexander Grothendieck as examples of people with lopsided profiles—stronger on “aesthetics” than they are on “fast pattern-recognition”.

I found it a thought-provoking hypothesis. I wish JonahS had written more.

There are more bullets to bite that I have personally thought of but never wrote up because they lean too much into "crazy" territory. Is there any place except lesswrong to discuss this anthropic rabbithole?

I remember reading this post, and really disliking it.

Then today, as I was reflecting on things, I recalled that this existed, and went back to read it. And this time, my reaction was instead "yep, that's pointing to the mental move that I've lost and that I'm now trying to relearn".

Which is interesting. Because that means a year or two ago, up till now, I was the kind of people who would benefit from this post; yet I couldn't get the juice out of it. I think a big reason is that while the description of the play/fun mental move is good and clear, the description of the opposite mental move, the one short-circuiting play/fun, felt very caricatural and fake.

My conjecture (though beware mind fallacy), is that it's because you emphasize "naive deference" to others, which looks obviously wrong to me and obviously not what most people I know who suffer from this tend to do (but might be representative of the people you actually met).

Instead, the mental move that I know intimately is what I call "instrumentalization" (or to be more memey, "tyranny of whys"). It's a move that doesn't require another or a social context (though it often includes internalized social judgements from others, aka superego); it only requires caring deeply about a goal (the goal doesn't actually matter that much), and being invested in it, somewhat neurotically.

Then, the move is that whenever a new, curious, fun, unexpected idea pop up, it hits almost instantly a filter: is this useful to reach the goal?

Obviously this filter removes almost all ideas, but even the ones it lets through don't survive unharmed: they get trimmed, twisted, simplified to fit the goal, to actually sound like they're going to help with the goal. And then in my personal case, all ideas start feeling like should, like weight and responsibility and obligations.

Anyway, I do like this post now, and I am trying to relearn how to use the "play" mental move without instrumentalizing everything away.

Thank you! I tried it on this [LW · GW] post and while the post itself is pretty short, the raw content that i get seems to be extremely long (making it larger than the o1 context window, for example), with a bunch of font-related information inbetween. Is there a way to fix this?

Disidentifying the consciousness from the body/shadow/subconscious it belongs to and is responsible for coordinating and speaking for, like many of the things some meditators do, wouldn't be received well by the shadow, and I'd expect it to result in decreased introspective access and control. So, psychonauts be warned.

A word of caution about interpreting results from these evals:

Sometimes, depending on social context, it's fine to be kind of a jerk if it's in the context of a game. Crucially, LLMs know that Minecraft is a game. Granted, the default Assistant personas implemented in RLHF'd LLMs don't seem like the type of Minecraft player to pull pranks out of their own accord. Still, it's a factor to keep in mind for evals that stray a bit more off-distribution from the "request-assistance" setup typical of the expected use cases of consumer LLMs.

Beauty of notation is an optimization target and so should fail as a metric, but especially compared to other optimization targets I’ve pushed on, in my experience it seems to hold up. The exceptions appear to be string theory and category theory and two failures in a field the size of math is not so bad.

Bywayeans are pretty censorious and scrupulous about violations of the NAP

Except against people who enjoy sunsets, apparently?

In retrospect, I am more pessimistic about AI having small amounts of niceness making humans live, and I now think that some amount of stronger alignment than pseudokindness is necessary to make humans survive with AI (but maybe not as strong as MIRI thinks), essentially because niceness to humans requires giving up opportunities to save compute on modeling the world, which is anti-incentivized by AI companies:

https://www.lesswrong.com/posts/xvBZPEccSfM8Fsobt/what-are-the-best-arguments-for-against-ais-being-slightly#wy9cSASwJCu7bjM6H [LW(p) · GW(p)]