LessWrong 2.0 Reader

I just donated $3000, for reasons based on Ryan's comment [LW(p) · GW(p)]. 

But humans don't seem to optimize for reward all that often!

You might be interested in an earlier discussion on whether "humans are a hot mess": https://www.lesswrong.com/posts/SQfcNuzPWscEj4X5E/the-hot-mess-theory-of-ai-misalignment-more-intelligent [LW · GW] https://www.lesswrong.com/posts/izSwxS4p53JgJpEZa/notes-on-the-hot-mess-theory-of-ai-misalignment [LW · GW]

If the last option is true, then the endpoints of this study, and most other studies on antidepressants in general, are entirely missing the forest for the trees.

This seems like the most likely scenario to me. I appreciate the need and desire to have a conclusive, defensible, formally rigorous answer to this question, but also, in some sense the answer very obviously has to end up being "Yes, but both 'depression' and 'antidepressant' are probably somewhat confused or overloaded categories that none of our available endpoints or diagnostic metrics demultiplex well enough to satisfy anyone or match patients' lived experiences."

I'm one patient who has been on bupropion for depression for ~6 years, and I truly cannot imagine a study that would convince me that it "didn't work" for me, in the ways that actually matter. The effects on my mind kick in sharply, scale smoothly with dose, decay right in sync with half-life in the body, and are clearly noticeable not just internally for my mood but externally in my speech patterns, reaction speeds, ability to notice things in my surroundings, short term memory, and facial expressions.

Lighthaven is quite different from the Lightcone Offices. Some key differences: 

• We mostly charge for things! This means that the incentives and social dynamics are a lot less weird and sycophantic, in a lot of different ways. Generally, both the Lightcone Offices and Lighthaven have strongly updated me on charging for things whenever possible, even if it seems like it will result in a lot of net-positive trades and arrangements not happening.
• Lighthaven mostly hosts programs, and doesn't provide office space for a ton of people. There is a set of permanent residents at Lighthaven, but we are talking about like 10-20 people including the Lightcone team, as opposed to the ~100+ people with approximately permanent access to the Lightcone Offices. As I mention in the fundraising post, I expect this set to grow very slowly, and I feel good about supporting everyone in this set (and would feel at the very least very conflicted and probably net bad about providing the same services to everyone who we supported via the Lightcone Offices)
• More broadly, Lighthaven is both a lot more curated, and a lot less insular. We have lots of big events here with people from adjacent communities and ecosystems, and I feel good about the marginal improvement to idea exchange and communication we make here. And then the people and programs we do support more consistently are things I feel good about.

I would add that all but the very worst versions of "human takeover" are much more recoverable than most versions of "AI takeover." And until and unless the human-in-charge makes sufficiently significant changes to their own biology and neurology that I'm not sure the term "human" still applies, they are susceptible to many more (and better understood) processes for changing their minds and morals. 

In other words: I don't want to end up in the human-takeover world, but I can imagine plausible future histories where that scenario gives way to an eventually-good outcome, in a way that seems much less likely for AI-takeover worlds.

Retrospectives are great, but I'm very confused at the juxtaposition of the Lightcone Offices being maybe net-harmful in early 2023 and Lighthaven being a priority in early 2025 [LW · GW]. Isn't the latter basically just a higher-production-value version of the former? What changed? (Or after taking the needed "space to reconsider our relationship to this whole ecosystem", did you decide that the ecosystem is OK after all?)

But even if the Thaba-Tseka Development Project is real & accurately described, what is the justification for focusing on this project in particular? It seems likely that James Ferguson focused on it b/c it was especially inept & hence it's not obviously representative of the World Bank's work in general.

I think I expect Earth in this case to just say no and not sell the sun? But I was confused at like 2 points in your paragraph so I don't think I understand what you're saying that well. I also think we're probably on mostly the same page, and am not that interested in hashing out further potential disagreements.

Also, mostly unrelated, maybe a hot take, but if you're able to get outcompeted because you don't upload, then the future you're in is not very good.

Cool. I misinterpreted your previous comment and think we're basically on the same page.

• Keep in mind that the only things that matter are things that need to get done before AGI. AGI can do the rest.
  • I acknowledge that there exist bets for long timelines, but I also think those things should be discounted compared to how promising they would be under a 20 year AGI timeline.
  • I also acknowledge that there’ll maybe be a centaur period of around 1 year after AGI. I don’t think this is that important a consideration as it’s unclear to me what it implies that the rest of the scenario doesn’t.

While I agree with this directionally, I'd warn people that you'd need both high confidence and also very good timing skills to make use of this well, rather than totally crashing, and also this will require reasonably good mental models on what you can safely assume AI does, and what AI won't be able to do over different time rates, so this advice only really has use for people already deeply thinking about AI.