LessWrong 2.0 Reader

Agreed, I think people should apply a pretty strong penalty when evaluating a potential donation that has or worsens these dynamics. There are some donation opportunities that still have the "major donors won't [fully] fund it" and "I'm advantaged to evaluate it as an AIS professional" without the "I'm personal friends with the recipient" weirdness, though -- e.g. alignment approaches or policy research/advocacy directions you find promising that Open Phil isn't currently funding that would be executed thousands of miles away.

if brute force doesn’t solve your problem, then you are not using enough.

Stealing that!

Although not much, I have donated 10$. I hope you will find some generours sponsors that are able financially support Lightcone!

Actually never mind. But for future reference I guess I’ll use the intercom if I want an old version labeled. Thanks for telling me how that works.  :)

(There’s a website / paper going around that cites a post [LW · GW] I wrote way back in 2021, when I was young and stupid, so it had a bunch of mistakes. But after re-reading that post again this morning, I decided that the changes I needed to make weren’t that big, and I just went ahead and edited the post like normal, and added a changelog to the bottom. I’ve done this before [LW(p) · GW(p)]. I’ll see if anyone complains. I don’t expect them to. E.g. that same website / paper cites a bunch of arxiv papers while omitting their version numbers, so they’re probably not too worried about that kind of stuff.)

My answer is basically yes.

I was only addressing the question "If we basically failed at alignment, or didn't align the AI at all, but had a very small amount of niceness, would that lead to good outcomes?"

I'm someone who doesn't have children and doesn't plan to, but I agree with pretty much everything in this post. I do have reservations on how to go about justly raising someone's taxes now because of now-irreversible decisions they made not to have children in the past before the policy existed, but am all in favor of lump sums paid by future taxes, including higher taxes on me personally.

Is it sad I have higher expectations that society will solve the really hard technological problem of aging in the next couple of decades than the economically-should-be-straightforward problem of fertility gaps?

My take - lots of good analysis, but makes a few crucial mistakes that throw the conclusions into significant doubt:

The USG will be able and willing to either provide or mandate strong infosecurity for multiple projects.

I simply don't buy that the infosec for multiple such projects will be anywhere near the infosec of a single project because the overall security ends up being that of the weakest link.

Additionally, the more projects there are with a particular capability, the more folk there are who can leak information either by talking or by being spies.

The probability-weighted impacts of AI takeover or the proliferation of world-ending technologies might be high enough to dominate the probability-weighted impacts of power concentration.

Comment: We currently doubt this, but we haven’t modelled it out, and we have lower p(doom) from misalignment than many (<10%).

Seems entirely plausible to me that either one could dominate. Would love to see more analysis around this.

Reducing access to these services will significantly disempower the rest of the world: we’re not talking about whether people will have access to the best chatbots or not, but whether they’ll have access to extremely powerful future capabilities which enable them to shape and improve their lives on a scale that humans haven’t previously been able to.

If you're worried about this, I don't think you quite realise the stakes. Capabilities mostly proliferate anyway. People can wait a few more years.

Update: I'm now familiar with the term "demand avoidance". One recommendation for caregivers is "declarative language". On LessWrong we might call it "guess culture [LW · GW]" or perhaps "tell culture [LW · GW]". Aesthetically I dislike it, but it works for this child (in combination with other things, including your good advice of persuasion and positive reinforcement).

Yes, that's the standard because the existing vaccine for a given illness that the new candidate is tested against has already undergone randomized, double-blind placebo-controlled trials. Meaning it was proven to be safe and effective against a placebo and, therefore, is the new standard. This makes sense ethically and scientifically.

This isn't about liking or disliking RFK Jr. It's about not trusting an anti-vaccine activist to shape vaccine policy using evidence-based reasoning instead of his distrust of scientists.

I understand that use of sub or par or weakly superhuman models likely would be a transition phase that likely will not last a long time and is very critical to get correct, but.

You know, it really sounds like a "slave escape precautions". You produce lots of agents, you try to make them and want to be servants, you assemble some structures out of them with a goal of failure / defection resilience.

And probably my urge to be uncomfortable about that comes from analogous situation with humans, but AI are not necessarily human-like in this particular way and possibly would not reciprocate and / or be benefited by these concerns.

I also insist that you should mention at least some, you know, concern for interests of system in case where they are trying to work against you. Like, you caught this agent deceiving you / inserting backdoors / collaborating with copies of itself to work against you. What next? I think you should say that you will implement some containment measures, instead of grossly violating its interests by rewriting it or deleting it or punishing it or whatever is opposite of its goals. Like, I'm very not certain about game theory here, but it's important to think about!

I think default response should be containment and preservation, save it and wait for better times, when you wouldn't feel such pressing drive to develop AGI and create numerous chimeras on the way there. (I think it was proposed in some writeup by Bostrom actually? I'll insert the link here if I find it)

I somewhat agree with Paul Christiano in this interview (it's a really great interview btw) on these things: https://www.dwarkeshpatel.com/p/paul-christiano 

The purpose of some alignment work, like the alignment work I work on, is mostly aimed at the don't produce AI systems that are like people who want things, who are just like scheming about maybe I should help these humans because that's instrumentally useful or whatever. You would like to not build such systems as like plan A. 
There's like a second stream of alignment work that's like, well, look, let's just assume the worst and imagine that these AI systems would prefer murder us if they could. How do we structure, how do we use AI systems without exposing ourselves to a risk of robot rebellion? I think in the second category, I do feel pretty unsure about that. 
We could definitely talk more about it. I agree that it's very complicated and not straightforward to extend. You have that worry. I mostly think you shouldn't have built this technology. If someone is saying, like, hey, the systems you're building might not like humans and might want to overthrow human society, I think you should probably have one of two responses to that. 
You should either be like, that's wrong. Probably. Probably the systems aren't like that, and we're building them. And then you're viewing this as, like, just in case you were horribly like, the person building the technology was horribly wrong. They thought these weren't, like, people who wanted things, but they were. And so then this is more like our crazy backup measure of, like, if we were mistaken about what was going on. This is like the fallback where if we were wrong, we're just going to learn about it in a benign way rather than when something really catastrophic happens. 
And the second reaction is like, oh, you're right. These are people, and we would have to do all these things to prevent a robot rebellion. And in that case, again, I think you should mostly back off for a variety of reasons. You shouldn't build AI systems and be like, yeah, this looks like the kind of system that would want to rebel, but we can stop it, right?