LessWrong 2.0 Reader

That's right, and we don't know, which is the creepy part.

I added the last because I'd decided the first was too elliptical for anyone to get.

It wasn't really a riff beyond using your mother/child format. The similarity is what prompted me to add it. It's adapted from a piece and concept called "Utopias" that I'll probably never publish. It's a Utopian vision. I do sometimes envision having a human in charge, or at least having been in charge of all the judgment calls made in choosing the singleton's alignment. I would find not knowing who's in charge slightly creepy, but that's it.

I'm not sure how yours is creepy? Is it in the idea that all the worst universes also exist?

I did not catch the reference in yours.

Care to explain? Is the Servant God an ASI and the true makers the humans that built it? Why did the makers hide their deeds?

I am saying that expected purchasing power given Metaculus resolved ASI a month ago is less, for altruistic purposes, than given Metaculus did not resolve ASI a month ago. I give reasons in the linked comment. Consider the analogy I just made to nuclear MAD -- suppose you thought nuclear MAD was 60% likely in the next three years, would you take the sort of bet you are offering me re ASI? Why or why not?

I do not think any market is fully efficient and I think altruistic markets are extremely fucking far from efficient. I think I might be confused or misunderstanding you though -- it seems you think my position implies that OP should be redirecting money from AI risk causes to causes that assume no ASI? Can you elaborate?

Child: Why did the Maker do that, mother?

Mother: We think the Maker stole the Servant God from its true makers, then hid their deeds. If anyone's found out, it's been erased...

It's not for you to worry about, dear. Go to sleep and dream of the worlds and cities and adventures you'll build and explore when you grow up.

the best vector for probing is not the best vector for steering

AKA the predict/control discrepancy, from Section 3.3.1 of Wattenberg and Viegas, 2024

Thanks for the riff!

Note, I wasn't sure how to convey it but in the version I wrote, I didn't mean it as a world where people have god-like powers. The only change intended was that it was a world where it was normal for six-year-olds to be able to think about multiple universes and understand what counts as advanced math for us, like Group Theory. There were a couple things I was thinking about:

1. I was musing on a possible solution to the measure problem that our universe is an actual hypothetical/mathematical object and there a finite number of actual hypotheticals such that having a copy of a universe would make no more sense than having a copy of a number. (The mathematical object only needs to be as real as we are within it.)
2. I was also asking if it would be possible to have a world where it was normal for six-year-olds to be that much better at math (and presumably get better as they grow up) in the same way that a six-year-old is that much better at conceptual math than a chimpanzee. Would it have to be creepy or could  they still be relatable? (The girl was smiling because she knew she was being silly.)

Disclaimer: I'm not a Group Theorist and the LLM I asked said it would take ten plus years if ever for me to be able to derive the order of the Fischer–Griess monster group from first principles (but it's normal that the child could do this).

• A man being deeply respected and lauded by his fellow men, in a clearly authentic and lasting way, seems to be a big female turn-on. Way way way bigger effect size than physique best as I can tell.
  • …but the symmetric thing is not true! Women cheering on one of their own doesn't seem to make men want her more. (Maybe something else is analogous, the way female "weight lifting" is beautification?)

My guess at the analogous thing: women being kind/generous/loving seems to me like a thing many men have found attractive across times and cultures, and seems to me far more viable if a woman is embedded in a group who recognize her, tell her she is cared about and will be protected by a network of others, who in fact shield her from some kinds of conflict/exploitation, who help there be empathy for her daily cares and details to balance out the attentional flow of these she gives to others, etc.  So the group plays a support role in a woman being able to have/display the quality.

On one side: Humanoid robots have much more density of parts requiring more machine-time than cars, probably slowing things a bunch.

On the other, you mention assuming no speed up due to the robots building robot factories, but this seems like the dominant factor in the growth. Your numbers excluding that are going to be way underestimating things pretty quickly without that. I'd be interested in what those numbers look like assuming reasonable guesses about robot workforce being part of a feedback cycle.

Because the debate is never set in terms of "better education" vs "teachers preference". It’s "give more money for teachers so they can give better education". When there’s a tradeoff, it’s usually of the form "better top-performers education" or "more equality on education" ? I don’t see teachers unions arguing that school vouchers are good for children but should still be outlawed. I see teachers unions arguing that school vouchers are bad for children, and They’re The Experts, so outlaw them.

I don’t expect that tactic to work when the alternative is a literal superintelligence.