Posts
Comments
Thanks for the ideas!
I am not sure which way you intended that sentence. Did you mean:
A. We want to shut down all AGI research everywhere by everyone, or
B. We want to shut down AGI research and we also want to shut down governments and militaries and spies
I assume you meant the first thing, but want to be sure!
We support A. Eliezer has been very clear about that in his tweets. In broader MIRI communications, it depends on how many words we have to express our ideas, but when we have room we spell out that idea.
I agree that current / proposed regulation is mostly not aimed at A.
As I mentioned in the post we are looking to hire or partner with a new spokesperson if we can find someone suitable. We don't think it will be easy to find someone great; it's a pretty hard job.
Seems right, thanks.
Gosh, I haven't really conducted a survey here or thought deeply about it, so this answer will be very off the cuff and not very 2024. Some of the examples that come to mind are the major media empires of, e.g. Brene Brown or Gretchen Rubin.
Yes.
Rob Bensinger has tweeted about it some.
Overall we continue to be pretty weak in on the "wave" side, having people comment publicly on current events / take part in discourse, and the people we hired recently are less interested in that and more interested in producing the durable content. We'll need to work on it.
Oh, yeah, to be clear I completely made up the "rock / wave" metaphor. But the general model itself is pretty common I think; I'm not claiming to be inventing totally new ways of spreading a message, quite the opposite.
Yup to all of that. :)
Your curiosity and questions are valid but I'd prefer not to give you more than I already have, sorry.
What are the artifacts you're most excited about, and what's your rough prediction about when they will be ready?
Due to bugs in human psychology, we are more likely to succeed in our big projects if we don't yet state publicly what we're going to do by when. Sorry. I did provide some hints in the main post (website, book, online reference).
how do you plan to assess the success/failure of your projects? Are there any concrete metrics you're hoping to achieve? What does a "really good outcome" for MIRI's comms team look like by the end of the year,
The only concrete metric that really matters is "do we survive" but you are probably interested some intermediate performance indicators. :-P
The main things I am looking for within 2024 are not as SMART-goal shaped as you are probably asking for. What I'd like to see is that that we've developed enough trust in our most recent new hires that they are freely able to write on behalf of MIRI without getting important things wrong, such that we're no longer bottlenecked on a few key people within MIRI; that we're producing high-quality content at a much faster clip; that we have the capacity to handle many more of the press inquiries we receive rather than turning most of them down; that we're better positioned to participate in the 'wave' shaped current event conversations.
I'd like to see strong and growing engagement with the new website.
And probably most importantly, when others in our network engage in policy conversations, I'd like to hear reports back that our materials were useful.
what does a "we have failed and need to substantially rethink our approach, speed, or personnel" outcome look like?
Failure looks like: still bottlenecked on specific people, still drowning in high-quality press requests that we can't fulfill even though we'd like to, haven't produced anything, book project stuck in a quagmire, new website somehow worse than the old one / gets no traffic, etc.
In this reply I am speaking just about the comms team and not about other parts of MIRI or other organizations.
We want to produce materials that are suitable and persuasive for the audiences I named. (And by persuasive, I don't mean anything manipulative or dirty; I just mean using valid arguments that address the points that are most interesting / concerning to our audience in a compelling fashion.)
So there are two parts here: creating high quality materials, and delivering them to that audience.
First, creating high quality materials. Some of this is down to just doing a good job in general: making the right arguments in the right order using good writing and pedagogical technique; none of this is very audience specific. There is also an audience-specific component, and to do well on that, we do need to understand our audience better. We are working to recruit beta readers from appropriate audience pools.
Second, delivering them to those audiences. There are several approaches here, most of which will not be executed by the comms team directly, we hand off to others. Within comms, we do want to see good reach and engagement with intelligent general audiences.
We think that most people who see political speech know it to be political speech and automatically discount it. We hope that speaking in a different way will cut through these filters.
At the start of 2024, the comms team was only me and Rob. We hired Harlan in Q1 and Joe and Mitch are only full time as of this week. Hiring was extremely labor-intensive and time consuming. As such, we haven't kicked into gear yet.
The main publicly-visible artifact we've produced so far is the MIRI newsletter; that comes out monthly.
Most of the rest of the object-level work is not public yet; the artifacts we're producing are very big and we want to get them right.
All of your questions fall under Lisa's team and I will defer to her.
A reasonable point, thank you. We said it pretty clearly in the MIRI strategy post in January, and I linked to that post here, but perhaps I should have reiterated it.
For clarity: we mostly just expect to die. But while we can see viable paths forward at all, we'll keep trying not to.
That phrase sounds like the Terminator movies to me; it sounds like plucky humans could still band together to overthrow their robot overlords. I want to convey a total loss of control.
In documents where we have more room to unpack concepts I can imagine getting into some of the more exotic scenarios like aliens buying brain scans, but mostly I don't expect our audiences to find that scenario reassuring in any way, and going into any detail about it doesn't feel like a useful way to spend weirdness points.
Some of the other things you suggest, like future systems keeping humans physically alive, do not seem plausible to me. Whatever they're trying to do, there's almost certainly a better way to do it than by keeping Matrix-like human body farms running.
I don't speak for Nate or Eliezer in this reply; where I speak about Eliezer I am of course describing my model of him, which may be flawed.
Three somewhat disjoint answers:
- From my perspective, your point about algorithmic improvement only underlines the importance of having powerful people actually get what the problem is and have accurate working models. If this becomes true, then the specific policy measures have some chance of adapting to current conditions, or of being written in an adaptive manner in the first place.
- Eliezer said a few years ago that "I consider the present gameboard to look incredibly grim" and while he has more hope now than he had then about potential political solutions, it is not the case (as I understand it) that he now feels hopeful that these solutions will work. Our policy proposals are an incredible longshot.
- One thing we can hope for, if we get a little more time rather than a lot more time, is that we might get various forms of human cognitive enhancement working, and these smarter humans can make more rapid progress on AI alignment.
Indeed! However, I'd been having stress dreams for months about getting drowned in the churning tidal wave of the constant news cycle, and I needed something that fit thematically with 'wave.' :-)
Writers at MIRI will primarily be focusing on explaining why it's a terrible idea to build something smarter than humans that does not want what we want. They will also answer the subsequent questions that we get over and over about that.
We want a great deal of overlap with Pacific time hours, yes. A nine-hour time zone difference would probably be pretty rough unless you're able to shift your own schedule by quite a bit.
Of course. But if it's you, I can't guess which application was yours from your LW username. Feel free to DM me details.
No explicit deadline, I currently expect that we'll keep the position open until it is filled. That said, I would really like to make a hire and will be fairly aggressively pursuing good applications.
I don't think there is a material difference between applying today or later this week, but I suspect/hope there could be a difference between applying this week and next week.
"Wearing your [feelings] on your sleeve" is an English idiom meaning openly showing your emotions.
It is quite distinct from the idea of belief as attire from Eliezer's sequence post, in which he was suggesting that some people "wear" their (improper) beliefs to signal what team they are on.
Nate and Eliezer openly show their despair about humanity's odds in the face of AI x-risk, not as a way of signaling what team they're on, but because despair reflects their true beliefs.
2. Why do you see communications as being as decoupled (rather, either that it is inherently or that it should be) from research as you currently do?
The things we need to communicate about right now are nowhere near the research frontier.
One common question we get from reporters, for example, is "why can't we just unplug a dangerous AI?" The answer to this is not particularly deep and does not require a researcher or even a research background to engage on.
We've developed a list of the couple-dozen most common questions we are asked by the press and the general public and they're mostly roughly on par with that one.
There is a separate issue of doing better at communicating about our research; MIRI has historically not done very well there. Part of it is that we were/are keeping our work secret on purpose, and part of it is that communicating is hard. To whatever extent it's just about 'communicating is hard,' I would like to do better at the technical comms, but it is not my current highest priority.
Re: the wording about airstrikes in TIME: yeah, we did not anticipate how that was going to be received and it's likely we would have wordsmithed it a bit more to make the meaning more clear had we realized. I'm comfortable calling that a mistake. (I was not yet employed at MIRI at the time but I was involved in editing the draft of the op-ed so it's at least as much on me as anybody else who was involved.)
Re: policy division: we are limited by our 501(c)3 status as to how much of our budget we can spend on policy work, and here 'budget' includes the time of salaried employees. Malo and Eliezer both spend some fraction of their time on policy but I view it as unlikely that we'll spin up a whole 'division' about that. Instead, yes, we partner with/provide technical advice to CAIP and other allied organizations. I don't view failure-to-start-a-policy-division as a mistake and in fact I think we're using our resources fairly well here.
Re: critiquing existing policy proposals: there is undoubtedly more we could do here, though I lean more in the direction of 'let's say what we think would be almost good enough' rather than simply critiquing what's wrong with other proposals.
Ditto.
I think that's pretty close, though when I hear the word "activist" I tend to think of people marching in protests and waving signs, and that is not the only way to contribute to the effort to slow AI development. I think more broadly about communications and policy efforts, of which activism is a subset.
It's also probably a mistake to put capabilities researchers and alignment researchers in two entirely separate buckets. Their motivations may distinguish them, but my understanding is that the actual work they do unfortunately overlaps quite a bit.
That's pretty surprising to me; for a while I assumed that the scenario where 10% of the population knew about superintelligence as the final engineering problem was a nightmare scenario e.g. because it would cause acceleration.
"Don't talk too much about how powerful AI could get because it will just make other people get excited and go faster" was a prevailing view at MIRI for a long time, I'm told. (That attitude pre-dates me.) At this point many folks at MIRI believe that the calculus has changed, that AI development has captured so much energy and attention that it is too late for keeping silent to be helpful, and now it's better to speak openly about the risks.
- What do you see as the most important messages to spread to (a) the public and (b) policymakers?
That's a great question that I'd prefer to address more comprehensively in a separate post, and I should admit up front that the post might not be imminent as we are currently hard at work on getting the messaging right and it's not a super quick process.
- What mistakes do you think MIRI has made in the last 6 months?
Huh, I do not have a list prepared and I am not entirely sure where to draw the line around what's interesting to discuss and what's not; furthermore it often takes some time to have strong intuitions about what turned out to be a mistake. Do you have any candidates for the list in mind?
- Does MIRI also plan to get involved in policy discussions (e.g. communicating directly with policymakers, and/or advocating for specific policies)?
We are limited in our ability to directly influence policy by our 501(c)3 status; that said, we do have some latitude there and we are exercising it within the limits of the law. See for example this tweet by Eliezer.
- Does MIRI need any help? (Or perhaps more precisely "Does MIRI need any help from the right kind of person with the right kind of skills, and if so, what would that person or those skills look like?")
Yes, I expect to be hiring in the comms department relatively soon but have not actually posted any job listings yet. I will post to LessWrong about it when I do.
I do not (yet) know that Nye resource so I don't know if I endorse it. I do endorse the more general idea that many folks who understand the basics of AI x-risk could start talking more to their not-yet-clued-in friends and family about it.
I think in the past, many of us didn't bring this up with people outside the bubble for a variety of reasons: we expected to be dismissed or misunderstood, it just seemed fruitless, or we didn't want to freak them out.
I think it's time to freak them out.
And what we've learned from the last seven months of media appearances and polling is that the general public is actually far more receptive to x-risk arguments than we (at MIRI) expected; we've been accustomed to the arguments bouncing off folks in tech, and we over-indexed on that. Now that regular people can play with GPT-4 and see what it does, discussion of AGI no longer feels like far-flung science fiction. They're ready to hear it, and will only get more so as capabilities demonstrably advance.
We hope that if there is an upsurge of public demand, we might get regulation/legislation limiting the development and training of frontier AI systems and the sales and distribution of the high-end GPUs on which such systems are trained.
Thanks, much appreciated! Your work is on my (long) list to check out. Is there a specific video you're especially proud of that would be a great starting point?
Feel free to send me a discord server invitation at gretta@intelligence.org.
I think your thesis is not super crisp, because this was an off the cuff post! And your examples are accordingly not super clear either, same reason. But there's definitely still a nugget of an idea in here.
It's something like, with the decentralization of both taking a position in the first place, and commenting on other people's positions, the lizardmen have more access to the people taking positions than they did in a world without social media. And lizardmen can and do serious damage to individuals in a seemingly random fashion.
Yup, seems legit. Our species does not have sufficient community-norm tech to be decentralized with full-mesh contact in groups larger than Dunbar's number.
Also, what a sentence I just wrote. What the fuck.
Do what you need to do to take care of yourself! It sounds like you don't choose to open up to your wife about your distress, for fear of causing her distress. I follow your logic there, but I also hope you do have someone you can talk to about it whom you don't fear harming, because they already know and are perhaps further along on the grief / acceptance path than you are.
Good luck. I wish you well.
Yes, that's correct, I was referring to the fable. I should probably have included a broader hint about that.