LessWrong 2.0 Reader

Harry's brain tried to calculate the ramifications and implications of this and ran out of swap space.

this is very relatable

I'm not sure!

Douglas Adams answered this long ago of course:

For instance, on the planet Earth, man had always assumed that he was more intelligent than dolphins because he had achieved so much—the wheel, New York, wars and so on—whilst all the dolphins had ever done was muck about in the water having a good time. But conversely, the dolphins had always believed that they were far more intelligent than man—for precisely the same reasons.

No, I don't recall any ethical concerns. Just basic concerns such as the difficulty of finding a boss that I'm comfortable with, having control over my hours, etc.

This is a confusing post from my perspective, because I think of LI as being about beliefs and corrigibility being about desires.

If I want my AGI to believe that the sky is green, I guess it’s good if it’s possible to do that. But it’s kinda weird, and not a central example of corrigibility.

Admittedly, one can try to squish beliefs and desires into the same framework. The Active Inference people do that. Does LI do that too? If so, well, I’m generally very skeptical of attempts to do that kind of thing. See here [LW · GW], especially Section 7. In the case of humans, it’s perfectly possible for a plan to seem desirable but not plausible, or for a plan to seem plausible but not desirable. I think there are very good reasons that our brains are set up that way.

I don't know, but I can say that after a lot of hours of Alexander lessons my posture and movement improved in ways that would be described as "having less muscle tension" and this having less tension happened in conjunction with various sorts of opening and being more awake and moving closer to PNSE.

If I understand you correctly, you want to create an unprecedentedly efficient and coordinated network, made out of intelligent people with goodwill, that will solve humanity's problems in theory and in practice? 

My 9yo has recently enjoyed Ender’s Game, Harry Potter, Hitchhiker’s Guide to the Galaxy, and What If. He recently asked to borrow my The Vital Question (it came up in conversation about abiogenesis) and he’s mostly following it so far but has occasional questions for me, we’ll see how far he gets or if he loses steam.

For non-books, he wanted to do Khan academy cosmology / astronomy, I think he did one big unit of Khan academy math before losing interest, he likes Eureka crates (little kits to build your own soap dispenser, rivet press, ukulele, whatever, they come once a month, good gift), lotsa video games, and he was doing DuoLingo Spanish every night (he has a streak, he’s a total sucker for gamification) but to my dismay decided to switch to the rather less practical DuoLingo Klingon. ¯\_(ツ)_/¯

Streamlined the registration page, and added a field to note that you want to join a Less Wrong team: https://www.unaging.com/unaging-system-2/

Yes, humanity would be more likely to survive if OpenAI never existed or if it closed tomorrow.

I wonder why no one else has answered. Am I stepping on some landmine I don't know about?

What would drastically increase P(survival) is stopping all large training runs. If that turns out to be impossible, then some labs might be "better" than the average lab in that if their model is the first one to become capable of extincting humanity, it might choose not to do that because of technical details in how the lab made the model. (I don't put a lot of hope in this possibility.)

To my knowledge, no one who is not employed at OpenAI and who is not an investor in OpenAI believes that OpenAI is one of these "better" labs. OK, that is not literally true, but it is unlikely that anyone who is not employed at OpenAI and who is not an investor in OpenAI and who believes OpenAI is one of the "better" labs can provide an argument longer than 50 words that you or I would consider rational and coherent.

But even if OpenAI is shut down tomorrow and everyone working there is permanently prevented from working in AI (a pleasant thought!) the AI enterprise would still be the thorniest danger facing humanity.