Announcing MIRI’s new CEO and leadership team

• Does MIRI need any help? (Or perhaps more precisely "Does MIRI need any help from the right kind of person with the right kind of skills, and if so, what would that person or those skills look like?")

Yes, I expect to be hiring in the comms department relatively soon but have not actually posted any job listings yet. I will post to LessWrong about it when I do.

That said, I'd be excited for folks who think they might have useful background or skills to contribute and would be excited to work at MIRI, to reach out and let us know they exist, or pitch us on why they might be a good addition to the team.

Thanks for this response!

Do you have any candidates for the list in mind?

I didn't have any candidate mistakes in mind. After thinking about it for 2 minutes, here are some possible candidates (though it's not clear to me that any of them were actually mistakes):

• Eliezer's TIME article explicitly mentions that states should "be willing to destroy a rogue datacenter by airstrike." On one hand, it seems important to clearly/directly communicate the kind of actions that Eliezer believes the world will need to be willing to take in order to keep us safe. On the other hand, I think this particular phrasing might have made the point easier to critique/meme. On net, it's unclear to me if this was a mistake, but it seems plausible that there could've been a way to rephrase this particular sentence that maintains clarity while reducing the potential for misinterpretations or deliberate attacks. 
• I sometimes wonder if MIRI should have a policy division that is explicitly trying to influence policymakers. It seems like we have a particularly unique window over the next (few? several?) months, where Congress is unusually-motivated to educate themselves about AI. Until recently, my understanding is that virtually no one was advocating from a MIRI perspective (e.g., alignment is difficult, we can't just rely on dangerous capability evaluations, we need a global moratorium and the infrastructure required for it). I think the Center for AI Policy is now trying to fill the gap, and it seems plausible to me that MIRI should either start their own policy team ( or assume more of a leadership role at the Center for AI Policy-- EG being directly involved in hiring, having people in-person in DC, being involved in strategy/tactics, etc. This is of course conditional on the current CAIP CEO being open to this, and I suspect he would be). 
• Perhaps MIRI should be critiquing some of the existing policy proposals. I think MIRI has played an important role in "breaking" alignment proposals (i.e., raising awareness about the limitations of a lot of alignment proposals, raising counterarguments, explaining failure modes). Maybe MIRI should be "breaking" some of the policy proposals or finding other ways to directly improve the ways policymakers are reasoning about AI risk. 

Again, it seems plausible to me that none of these were mistakes; I'd put them more in the "hm... this seems unclear to me but seems worth considering" category.

• Does MIRI also plan to get involved in policy discussions (e.g. communicating directly with policymakers, and/or advocating for specific policies)?

We are limited in our ability to directly influence policy by our 501(c)3 status; that said, we do have some latitude there and we are exercising it within the limits of the law. See for example this tweet by Eliezer.

To expand on this a bit, I and a couple others at MIRI have been spending some time syncing up and strategizing with other people and orgs who are more directly focused on policy work themselves. We've also spent some time chatting with folks in government that we already know and have good relationships with. I expect we'll continue to do a decent amount of this going forward. 

It's much less clear to me that it makes sense for us to end up directly engaging in policy discussions with policymakers as an important focus of ours (compared to focusing on broad public comms), given that this is pretty far outside of our area of expertise. It's definitely something I'm interested in exploring though, and chatting about with folks who have expertise in the space.

I think I got a lot out of not knowing anything about the broader rationalist community while reading the sequences. I think that consideration weighs against.

I like the idea!

For Rational Animations, there's no problem if you do that, and I generally don't see drawbacks.

Perhaps one thing to be aware of is that some of the articles we'll animate will be slightly adapted. Sorting Pebbles and The Power of Intelligence are exactly like the original. The Parable of The Dagger has deletions of words such as "replied" or "said" and adds a short additional scene at the end. The text of The Hidden Complexity of Wishes has been changed in some places, with Eliezer's approval, mainly because the original has some references to other articles. In any case, when there are such changes, I write it in the LW post accompanying the videos.

If you just had to pick one, go for The Goddess of Everything Else. 

Here's a short list of my favorites.

In terms of animation: 

- The Goddess of Everything Else
- The Hidden Complexity of Wishes
- The Power of Intelligence

In terms of explainer:

- Humanity was born way ahead of its time. The reason is grabby aliens. [written by me]
- Everything might change forever this century (or we’ll go extinct). [mostly written by Matthew Barnett]

Also, I've sent the Discord invite.

I do not (yet) know that Nye resource so I don't know if I endorse it.

That makes sense, I would never ask for such an endorsement; I don't think it would help MIRI directly, but Soft Power is one of the most influential concepts among modern international relations experts and China experts, and it's critical for understanding the environment that AI safety public communication takes place in (e.g. if the world is already oversaturated with highly professionalized information warfare [LW · GW] then that has big implications e.g. MIRI could be fed false data to mislead them into believing they are succeeding at describing the problem when in reality the needle isn't moving).

I think in the past, many of us didn't bring this up with people outside the bubble for a variety of reasons: we expected to be dismissed or misunderstood, it just seemed fruitless, or we didn't want to freak them out.

I think it's time to freak them out.

That's pretty surprising to me; for a while I assumed that the scenario where 10% of the population knew about superintelligence as the final engineering problem was a nightmare scenario e.g. because it would cause acceleration. I even abstained from helping Darren McKee with his book attempting to describe the AI safety problem to the public, even though I wanted to, because I was worried about contributing to making people more capable of spreading AI safety ideas. 

If MIRI has changed their calculus on this, then of course I will defer to that since I have a sense of how far outside my area of expertise it is. But it's still a really big shift for me.

We hope that if there is an upsurge of public demand, we might get regulation/legislation limiting the development and training of frontier AI systems and the sales and distribution of the high-end GPUs on which such systems are trained.

I'm not sure what to make of this; AI advancement is pretty valuable for national security (e.g. allowing hypersonic nuclear missiles to continue flying under the radar if military GPS systems are destroyed or jammed/spoofed) and the balance of power between the US and China in other ways [LW · GW], similar to nuclear weapons; and if public opinion turned against nuclear weapons during the 1940s and 1950s, back when democracy was stronger, I'm not sure if it would have had much of an effect (perhaps it would have pushed it underground [LW · GW]). I'm still deferring to MIRI on this pivot and will help other people take the advice in this comment, and I found this comment really helpful and I'm glad that big pivots are being committed to, but I'll also still be confused and apprehensive.

Don't forget lc had a 624 karma post on it on April 4th 2022. [LW · GW]

My read was that his comment was in response to this part at the end of the post:

There’s a lot more we hope to say about our new (and still evolving) strategy, and about our general thinking on the world’s (generally very dire) situation. But I don’t want those announcements to further delay sharing the above updates, so I’ve factored our 2023 strategy updates into multiple posts, beginning with this one.

How so: "There’s a lot more we hope to say about our new (and still evolving) strategy, and about our general thinking on the world’s (generally very dire) situation. But I don’t want those announcements to further delay sharing the above updates, so I’ve factored our 2023 strategy updates into multiple posts, beginning with this one."

Unfortunately, I do not have a long response prepared to answer this (and perhaps it would be somewhat inappropriate, at this time), however I wanted to express the following:

They wear their despair on their sleeves [? · GW]? I am admittedly somewhat surprised by this. 

Quickly chiming in to add that I can imagine there might be some research we could do that could be more instrumentally useful to comms/policy objectives. Unclear whether it makes sense for us to do anything like that, but it's something I'm tracking.