Catching the Eye of Sauron

Refusing to engage billionaires on twitter - especially ones that are sufficiently open to being convinced that they will drop $44 billion for something as pedestrian as a social media company.

What on earth are you talking about? In the hyperlink you mention he is engaging him; he's just stating the brute fact that it would be really hard to deploy the money, which is something true and necessary for someone to understand if they were to try this. Doing the standard political thing of just grifting and assuring him that all he needs to do is pour money into the field would be unhelpful.

Notice the last time that someone "engaged" Elon Musk we got OpenAI. It is empirically extraordinarily easier to pour gasoline on the problem with barrels of money by accelerating race dynamics than it is to get meaningful work done.

Related: The question "Where is a comprehensive, well-argued explanation of Eliezer's arguments for AI risk, explaining all lingo, spelling out each step, referencing each claim, and open to open peer review" apparently has no answer? Cause a lot of people have been asking. They see the podcast, and are concerned, but unconvinced, and want to read a solid paper or book chapter, or the equivalent in website form. Something you can read over the course of a few hours, and actually have the questions answered. Something you can cite and criticise, and have Eliezer accept that this is a good version to attack. No handwaving, or vaguely referencing online texts without links, or concepts that are mentioned everywhere and never properly explained with hindsight, no pretending a step is trivial or obvious when it simply is not. But all the arguments in it. All the data in it.

I will say that I thought Connor Leahy's talk on ML Street Talk was amazing and that we should if possible make Connor go on Lex Fridman?

The dude looks like a tech wizard and is smart, funny, charming and a short timeline doomer. What else do you want?

Anyway we should create a council of charming doomers or something and send them at the media, it would be very epic. (I am in full agreement with this post btw)

I hard disagree with your point about the fedora

My true rejection is that we shouldn't be obsessing over people's appearances and self-expression, we shouldn't be asking people to be less than themselves. This is not truth-seeking. It gives off the vibe of fakeness and cults and of your mom telling you not to go out dressed like that.

My more principled rejections
- It overly focuses signalling on what some people on the internet care about over what the average american / person in political power / AI researcher cares about.
- I'm suspect yudkowsky is aware that the stereotypes of the fedora, and wears it anyways, perhaps to reclaim it as a celebration of smartness, perhaps to countersignal that he doesn't care what the sneer-clubbers think, perhaps to 'teach in a clown suit'.
- You don't win the meme war by doing the conventional media stuff good enough. You have to do something out of the distribution, and perhaps "low status" (to some). The Kardashians pioneered a new form of tv. Mr Beast studied a lot and pioneered a new form of media business. I should write a longer post on this. In any case, let's keep EA and rationality weird, it's one of the few edges we have.

+1

My background is extremely relevant here and if anybody in the alignment community would like help thinking through strategy, I'd love to be helpful.

Eliezer, or somebody better at talking to humans than him, needs to go on conservative talk shows - like, Fox News kind of stuff - use conservative styles of language, and explain AI safety there. Conservatives are intrinsically more likely to care about this stuff, and to get the arguments why inviting alien immigrants from other realms of mindspace into our reality - which will also take all of our jobs - is a bad idea. Talk up the fact that the AGI arms race is basically as bad as a second cold war only this time the "bombs" could destroy all of human civilization forever, and that anyone who develops it should be seen internationally as a terrorist who is risking the national security of every country on the planet.

To avoid the topic becoming polarized, someone else should at the same time go on liberal talk shows and explain how unaligned AGI, or even AGI aligned to some particular ideological group, is the greatest risk of fascism in human history and could be used to permanently lock the planet into the worst excesses of capitalism, fundamentalism, oppression, and other evils, besides the fact that (and this is one of the better arguments for abortion) it is immoral to bring a child (including AGI) into the world without being very sure that it will have a good life (and most people will not think that being addicted to paperclips is a good life).

I think you're right about the problem, but wrong about the solution. Doing all of the additional things you mention, but with the same communication strategy, is going to produce polarization. As it happens, I just wrote a post about this [LW · GW] yesterday.

If we get half the experts on board but cause the other half to dig in their heels and talk nonsense out of sheer irritation, the public and public policy will be gridlocked, like they are on climate change and pretty much every other red vs. blue issue. We need faster action, and the case is strong enough to get it.

Eliezer's approach on those podcasts was so bad that it's doing much more harm than good. Every reaction I've heard from outside the rationalist community has been somewhere from bad to actively irritated with the whole topic and the whole community.

Yudkowsky needs to step aside as the public face of AGI alignment, or he needs to get much better at it, quickly. I love Eliezar and value his work tremendously, but this is not his area, and he's using an approach that is basically known to produce polarization. And I can't imagine where he'd find the emotional energy to get good at this, given his current state of despair.

On the other hand, somebody needs to do it, and he's being given opportunities. I don't know if he can hand those interview requests off to someone else.

I would not have come to trust a socially-able-Eliezer. He's pure. Let him be that.

Existing rockstar researchers 

Eliezer explains in the following link (and in AGI ruin) why he thinks this is unlikely to work. It would certainly increase the "prestige" of AI alignment to have well known faces here, but that does not necessarily lead to good things. 

I agree the goal here should be effective memetics. Does Yud currently have a PR team? I am assuming the answer no.

Refusing to engage billionaires on twitter - especially ones that are sufficiently open to being convinced that they will drop $44 billion for something as pedestrian as a social media company. 

This one isn't obvious to me - having billionaires take radical action can be very positive or very negative (and last time this was tried on Elon he founded OpenAI!)

I'm surprised this has this many upvotes. You're taking the person that contributed the most to warning humanity about AI x-risks, and are saying what you think they could have done better in what comes across as blamy to me. If you're blaming zir, you should probably blame everyone. I'd much rather if you wrote what people could have done in general rather than targeting one of the best contributors.

I can't agree more with the post but I would like to note that even the current implementation is working. It definitely grabbed people's attention. 

My friend who never read LW writes in his blog about why we are going to die. My wife who is not a tech person and was never particularly interested in AI gets TikToks where people say that we are going to die.

So far it looks like definitely positive impact overall. But it's early to say, I am expecting some kind of shitshow soon. But even shitshow is probably better than nothing.

"You are, whether you like it or not, engaged in memetic warfare - and recent events/information make me think this battle isn't being given proper thought"

I'd like to chime in on the psychological aspects of such a warfare. I suggest that a heroic mindset will be helpful in mustering courage, hope and tenacity in fighting this cultural battle. In the following I will sketch out both a closer view of the heroic mindset, as well as a methodology for achieving it. 

A) On the heroic mindset
Heroism is arguably "altruistic risk, resilience, and fundamental respect for humanity". One could see it as part of manliness, but one could also have gender-inclusive versions of heroism, for example the rationalist hero, as per Eliezer Yudkowsky's reference to The World of Null-A-science fiction-series [LW · GW]. The key aspect for me is that the hero is willing to take initiative, to take a stand, is courageous, fights the good cause, doesn't become cynical, isn't a coward - and many other good qualities.

B) On achieving the heroic mindset
The Oxford Character project focuses on developing virtue, and has summarized some key findings here. One of the strategies for character development mentioned is "engagement with virtuous exemplars". In other words: imitating or taking on other people as role models - perhaps even identifying with the role model. Linda Zagzebski - who has written on virtue epistemology - has also written a book on this type of virtue development called "Exemplarist Moral Theory". Recommended. 

I believe that if we find some good role-models, for example the Null-A heroes, or others like Nelson Mandela, Winston Churchill, or Eleanor Roosevelt, we can identify with them, and thereby access our own resources. One practical way of going about this is through Todd Herman's The Alter Ego Effect: The Power of Secret Identities to Transform Your Life. [2019]. We basically take on a heroic identity, and go through our day from that frame of reference. This can be majorly empowering.

In summary, the key to taking on a heroic mindset lies in shifting one's identity to a heroic identity. And we can do that in practice through Zagzebski's or perhaps even more hands-on: through Todd Herman's work. 

By doing this, we might gain more fighting spirit and willingness to try to be actors and movers, not just "understanders" of this world and this situation.

Agree for the need of some of those things, though I wouldn't say things like e.g. Eliezer not giving any proposals to Musk in that Tweet is a sign of "refusing to engage". Last time Elon Musk tried to do something about AI safety, Open AI was born. I think it makes sense to hold back from giving more ideas to the one man whose sole attempt at solving the problem was also the one thing that most of all made the problem a lot worse in the last year.

The discussion in the comments is extremely useful and we've sorely needed much more of it. I think we need a separate place purely for sharing and debating our thoughts about strategies like this, and ideally also working on actual praxis based on these strategies. The ideal solution for me would be a separate "strategy" section on LessWrong or at least a tag, with much weaker moderation to encourage out-of-the-box ideas. So as not to pass the buck I'm in the process of building my own forum in the absence of anything better.

Some ideas for praxis I had, to add to the ones in this post and the comments: gather a database of experiences people have had of actually convincing different types of people of AI risk, and then try to quantitatively distill the most convincing arguments for each segment; proofreading for content expected to be mass consumed - this could have prevented the Time nukes gaffe; I strongly believe a mass-appeal documentary could go a long way to alignment-pilling a critical mass of the public. It's possible these are terrible ideas, but I lack a useful place to even discuss them.

The easiest point to make here is Yud's horrible performance on Lex's pod. It felt like no prep and brought no notes/outlines/quotes??? Literally why?

Millions of educated viewers and he doesn't prepare..... doesn't seem very rational to me. Doesn't seem like systematically winning to me.

Yud saw the risk of AGI way earlier than almost everyone and has thought a lot about it since then. He has some great takes and some mediocre takes, but all of that doesn't automatically make him a great public spokesperson!!!

He did not come off as convincing, helpful, kind, interesting, well-reasoned, humble, very smart, etc.

To me, he came off as somewhat out of touch, arrogant, weird, anxious, scared, etc. (to the average person that has never heard of Yud before the Lex pod)

Very well put, and I couldn't agree more with this. I've been reading and thinking more and more about the AI situation over the past year or so, starting when that AI researcher at Google became convinced that he had created a conscious being. Things are now accelerating at a shocking pace, and what once seemed like speculation that wasn't immediately relevant is now crucially urgent. Time is of the essence. Moreover, I'm becoming increasingly convinced that AI containment, if it is achieved, will be done through political solutions rather than technological solutions. Things are just moving way too fast, and I don't see how technical alignment will keep up when the pool of alignment researchers is so tiny compared to the enormous number of AI capabilities researchers.

For those of us deeply worried about AI risk, we're going to have to prepare for a rapid change in the discourse. Public persuasion will be crucial, as if we win it will be achieved by a combination of public persuasion and effective wielding of the levers of power. This means a paradigm shift in how capital-R Rationalists talk about this issue. Rationalists have a very distinctive mode of discourse which, despite having undeniable benefits, is fundamentally incongruent with more typical modes of thinking. We need to be willing to meet people where they are at, empathize with their concerns (including people worried about AI taking their jobs or making life meaningless - this seems to be quite common), and adopt non-Rationalist methods of persuasion and effective use of power that are known to be effective. Memetic warfare, one could call it. This will probably feel very dirty to some, and understandably so, but the paradigm has shifted and now is the time.

The methods of Rationality can still be very useful in this - it's an effective way to interrogate one's own assumptions and preexisting biases. But people have to be willing and able to use these methods in service of effective persuasion. Keeping our eyes on the prize will also be crucial - if this new limelight ends up getting used to advance other popular Rationalist causes and viewpoints such as atheism and wild animal suffering, I do not see how this could possibly go well.

With the TIME article, Bing's AI's aggression and Elon Musk's every other tweet being about AI safety and STILL nothing happening in the media, I think it's going to take something major to create the spark we need and then fan it into a real fire. 

The best idea I've had so far is a letter writing campaign in the form of carefully worded emails from all of us here on LW to literally anyone with a platform, but I feel we've got to be able to do better than that. 

Normally I wouldn't advocate such a thing, but if we can't convince people of the real big bag AI coming to get them, then the best thing we can do might be to create a fake and more tangible one. 

"Misaligned AGI flying nanobots up our noses in 5 years" - no headlines

"AI takes control of Boston Dynamics robot and kills a man because it's evil" - HEADLINES 

Journalists are pretty much allowed to get away with straight up lying, so if anyone smarter than me could make a fake story that would actually resonate with people hit the news, it'd be something I'd put a lot of hope in.