Yudkowsky vs Trump: the nuclear showdown.

he [Trump] is definitely weird

This reminds me of Paul Graham's essay about weird programming languages. Languages designed for people less smart than you seem dumb and powerless. Those designed for people smarter than you seem weird.

If someone seems weird I take that as a sign they may be playing a higher level game. (They may just be weird of course).

A lot of people think Trump is stupid and has low impulse control. Consider though that his uncle was a professor of physics and his sister was a judge. His father was too poor to go to college but showed every indication of being a very capable person. It seems unlikely he is over-endowed with low IQ genes.

How many people have strong opinions about Trump without much data to base that on? How many have read his books?

Read his account of how he worked his way around the bullyish teacher Theodore Dobias - a strategy that, if you believe it, required a lot of restraint and finesse.

I'm a right winger and I totally disagree with this comment.

For me, conservatism is about willingness to face up to the hard facts about reality. I'm just as cosmopolitan in my values as liberals are--but I'm not naive about how to go about achieving them. My goal is to actually help people, not show all my friends how progressive I am.

In practice I think US stability is extremely important for the entire world. Which means I'm against giving impulsive people the nuclear codes, and I'm also against Hillary Clinton's "invade the world, invite the world" foreign policy.

Also: I don't like Yudkowsky, but I would like him and the people in his circle to take criticism seriously, so... could we maybe start spelling his name correctly? It ends in a y. (I think Yudkowsky himself is probably a lost cause, but there are a lot of smart, rational people in his thrall who should not be. And many of them will take the time to read and seriously evaluate critical arguments if they're well-presented.)

I'm a left-libertarian and i mostly disagree with this comment, but i upvoted it because it's very clear and respectful.

I agree that politics discussions are better suited for other rationality-sphere sites, not LW.

While I'm horrified by your values, I thank you for the clear exposition. And I think you made, although implicitly, an interesting point: outside of our stable equilibrium, things are a generally a net negative if you factor in the welfare of all humanity. But if you consider only US' well being, things might be a net positive.
This I believe is so far from Yudkowsky's set of values that it was impossible for him to envision.

LW isn't a place to argue about politics, so I'm not going to go into why we have the values that we have here.

Well, an easy guess would be that you believe the world to be a zero-sum game: if that's the case, then better have your team win big at the expense of everyone else than having everybody live on the scraps.

Like, reading through Yudkowsky's stuff, his LW writings and HPMOR, there is the persistent sense that he is 2 guys.

One guy is like "Here are all of these things you need to think about to make sure that you are effective at getting your values implemented". I love that guy. Read his stuff. Big fan.

Other guy is like "Here are my values!" That guy...eh, not a fan. Reading him you get the idea that the whole "I am a superhero and I am killing God" stuff is not sarcastic.

It is the second guy who writes his facebook posts.

Yes, I agree with this sentiment and am relieved someone else communicated it so I didn't have to work out how to phrase it.

I don't share (and I don't think my side shares), Yudkowsky's fetish for saving every life. When he talks about malaria nets as the most effective way to save lives, I am nodding, but I am nodding along to the idea of finding the most effective way to get what you want done, done. Not at the idea that I've got a duty to preserve every pulse.

I don't think Yudkowsky think malaria nets are the best use of money anyway, even if they are in the short term the current clearest estimate as to where to put your money in in order to maximise lives saved. In that sense I don't think you disagree with him, he doesn't fetishize preserving pulses in the same way that you don't. Or at least, that's what I remember reading. First thing I could find corroborating that model of his viewpoint is his interview with Horgan.

There is a conceivable world where there is no intelligence explosion and no superintelligence. Or where, a related but logically distinct proposition, the tricks that machine learning experts will inevitably build up for controlling infrahuman AIs carry over pretty well to the human-equivalent and superhuman regime. Or where moral internalism is true and therefore all sufficiently advanced AIs are inevitably nice. In conceivable worlds like that, all the work and worry of the Machine Intelligence Research Institute comes to nothing and was never necessary in the first place, representing some lost number of mosquito nets that could otherwise have been bought by the Against Malaria Foundation.

There’s also a conceivable world where you work hard and fight malaria, where you work hard and keep the carbon emissions to not much worse than they are already (or use geoengineering to mitigate mistakes already made). And then it ends up making no difference because your civilization failed to solve the AI alignment problem, and all the children you saved with those malaria nets grew up only to be killed by nanomachines in their sleep. (Vivid detail warning! I don’t actually know what the final hours will be like and whether nanomachines will be involved. But if we’re happy to visualize what it’s like to put a mosquito net over a bed, and then we refuse to ever visualize in concrete detail what it’s like for our civilization to fail AI alignment, that can also lead us astray.)

I think that people who try to do thought-out philanthropy, e.g., Holden Karnofsky of Givewell, would unhesitatingly agree that these are both conceivable worlds we prefer not to enter. The question is just which of these two worlds is more probable as the one we should avoid. And again, the central principle of rationality is not to disbelieve in goblins because goblins are foolish and low-prestige, or to believe in goblins because they are exciting or beautiful. The central principle of rationality is to figure out which observational signs and logical validities can distinguish which of these two conceivable worlds is the metaphorical equivalent of believing in goblins.

I think it’s the first world that’s improbable and the second one that’s probable. I’m aware that in trying to convince people of that, I’m swimming uphill against a sense of eternal normality – the sense that this transient and temporary civilization of ours that has existed for only a few decades, that this species of ours that has existed for only an eyeblink of evolutionary and geological time, is all that makes sense and shall surely last forever. But given that I do think the first conceivable world is just a fond dream, it should be clear why I don’t think we should ignore a problem we’ll predictably have to panic about later. The mission of the Machine Intelligence Research Institute is to do today that research which, 30 years from now, people will desperately wish had begun 30 years earlier.

Also, on this:

Yes, electing Hillary Clinton would have been a better way to ensure world prosperity than electing Donald Trump would. That is not what we are trying to do. We want to ensure American prosperity.

Especially here, I'm pretty sure Eliezer is more concerned about general civilisational collapse and other globally negative outcomes which he sees as non-trivially more likely with Trump as president. I don't think this is as much of a difference in values and specifically differences with regards to how much you each value each level of the concentric circles of the proximal groups around you. At the very least, I don't think he would agree that a Trump presidency would be likely to result in improved American prosperity over Clinton.

I just want to point out that Yudkowsky is making the factual mistake of modeling us as being shitty at achieving his goals, when in truth we are canny at achieving our own.

I think this is probably not what's going on, I honestly think Eliezer is being more big picture about this, in the sense that he is concerned more about increased probability of doomsday scenarios and other outcomes unambiguously bad for most human goals. That's the message I got from his facebook posts anyway.

Indeed, the "I am a superhero and I am killing God" stuff, as you put it, isn't sarcastic. He really is trying to make saving every single living human mind he can his life's work...

Hey, I stumbled on this comment and I'm wondering if you've updated on whether you consider Trump/Republicans a threat to America's interests in light of the January 6th insurrection.

One guy is like "Here are all of these things you need to think about to make sure that you are effective at getting your values implemented". I love that guy. Read his stuff. Big fan.

Other guy is like "Here are my values!" That guy...eh, not a fan. Reading him you get the idea that the whole "I am a superhero and I am killing God" stuff is not sarcastic.

It's almost like epistemic and instrumental rationality are two different things.....

As I understand it you are criticizing Yudkowski's ideology. But MrMind wants to hear our opinion on whether or not Scott and Yudkowski's reasoning was sound, given their ideologies.

The whole "most variations from the equilibria are disasters", only really works if you share my guy's mania about valuing the other team's welfare

You know , there was one other guy who wasn't preoccupied at all about the other team's welfare , that guy was John Von Neumann and were he able to have it his way he would have cold bloodedly killed 600 millions people between USSR and China in 1955 when US had B52s and thermonuclear bombs ironed out , while he could have used his intelligence and technical wisdom to deescalate tension with the ultimate goal of getting rid of nuclear weapons altogether . Irony of the ironies he died relatively young because of a cancer probably developed working on the bomb , exactly like his soviets counterparts...he'd have had more chances of surviving if american , soviet and chinese researchers would have been able to talk to each other and exchange informations on potential life saving treatments . Besides that I don't even mention the damage that wiping out 600 millions people would have done to the world's economy , the world would have been a very different place if Von Neumann succeeded in acting his personal version of the final solution

There is only one team and that's team humanity , the prostate cancer which kills a russian citizen is the same identical disease which would take your life if you're unlucky enough to develop one , so given that ever since we (almost) stopped killing each other over land we enjoyed a prosperity which has no precedent in the history of our specie and it is mostly correlated with the fact that there are more humans around to solve our common problems , so how about we keep it that way? Also how about we increase the number of humans around and we lift them from poverty so they'd be able to contribute to the economy and together find a solution to our common problems (energy crisis , diseases , aging , AI) ?

Also I agree with you that the "preserve every pulse" kind of thinking could lead to an impractical situation , but I also think that the correct approach for this issue is the "in medio stat virtus" approach being something like "If you create damages to society which are greater than your contribution to it for a continued period of e.g. 5 years" your life would not be worth preserving

the only danger to us is a nuclear war (meaning Russia)

Such danger only exists because Russian people are possibly even worse than americans at spotting con-artists and calling them out on their BS (europeans seems to be better than anyone else at doing this , maybe because they suffered so much in the past when they failed to do it) , so they praise and elevate Putin as a modern day czar because their life conditions sensibly improved with respect to the Yeltsin years , but they fail to see how much power and wealth is concentrated in the hands of their president who is able to casually steal 1 billion dollars from the State budget to build a private palace on the Black Sea . It is the support of Russian people which enables Putin to threaten the world with the apocalyptic scenario of a nuclear war , but as the Arab Spring proved such support is not destined to go on forever , dictators get only deposed when the people of that country collectively think that their lives would be better without him....if all the westerners who waste time every day watching Netflix or playing video games dedicated that time to talk with their Russian counterparts through the internet , provide them information which would not be otherwise available given the regime's propaganda and yes , even send them 0.5BTC whenever they can to show support and compassion a dictator like Putin would be deposed and hanged within 6 months , much to the relief of people living in adjacent countries (whose suffering you seem to ignore and perhaps more importantly role in the world's economy you seem to ignore) and the rest of the world

Given our invincible military

This honestly seems a phrase straight off a propaganda poster , are you even aware of the costs in terms of brainpower and capital which are wasted every year on the military? Think of what could be accomplished if such resources were redirected towards research and basic research ( AI , FAI , WBE , nuclear fusion , brain understanding , consciousness understanding..)

To spell it out: I don't share (and I don't think my side shares), Yudkowski's fetish for saving every life. When he talks about malaria nets as the most effective way to save lives, I am nodding, but I am nodding along to the idea of finding the most effective way to get what you want done, done. Not at the idea that I've got a duty to preserve every pulse.

So are you seriously claiming that you can't see the correlation between number of humans alive on the Earth and average quality of life and progress achieved by our specie?

Putin will thug around his neighbors

Yeah right , because Putin putting his hands on the mineral rich and fertile soils of Ukraine is a totally desirable outcome for the world's economy

Yes, electing Hillary Clinton would have been a better way to ensure world prosperity than electing Donald Trump would. That is not what we are trying to do. We want to ensure American prosperity

And that , my friend is the line of thinking which caused the outbreak of every war in the history of our specie , also electing a guy who spent 5 millions dollars to have his bathroom completely gold plated seems the best way to ensure american prosperity /s , also you continuously mention Putin , Trump is the presidential candidate who resembles him the most , except for maybe one thing that would sure impress the donald the first time he'll meet him , Putin speaks a fluent english with a marked BBC accent , you could almost say that between the 2 , the Russian from St. Petersburg has the better words

Trump will (probably not) build a wall between us and Mexico

Mark my words he will , and he will channel public money through his companies in order to build it , he'll try to pull it off in the 2 years before the midterm elections...that would be something very Putinesque of him (see the similarities between the two are recurrent )

Saying "we haven't had a nuclear exchange with Russia yet, therefor our foreign policy and diplomatic strategy is good" is an obvious fallacy.

That's a strawman. EY isn't saying that our foreign policy is good.

None of this is a much my strongly held beliefs as my attempt to find flaw with the "nuclear blackmail" argument.

I don't understand. Could you correct the grammar mistakes or rephrase that?

The way I understand the argument isn't that the status quo in the level B game is perfect. It isn't that Trump is a bad choice because his level B strategy is taking too much risk and therefore bad. I understand the argument as saying: "Trump doesn't even realize that there is a level B game going on and even when he finds out he will be unfit to play in that game".

This is a really good comment, and I would love to hear responses to objections of this flavour from Eliezer etc.

Saying "we haven't had a nuclear exchange with Russia yet, therefor our foreign policy and diplomatic strategy is good" is an obvious fallacy. Maybe we've just been lucky.

I mean it's less about whether or not it's good as much as it is trying to work out the likelihood of whether policies resulting from Trump's election are going to be worse. You can presuppose that current policies are awful and still think that Trump is likely to make things much worse.

I also don't see that the status quo is keeping the world all that safe, with a proxy war between the US and Russia in Syria.

That's not the only argument. The fact that Trump suggested that it's fine that the Saudi's (and others) get nukes. That statement gave him no electoral advantage but he still made it.

Can you really believe that when the right challenges the status quo, priors are that they almost certainly wrong, but when progressives challenge the status quo they are almost certainly correct? This is extremely motivated reasoning and totally inconsistent.

I don't think that's the case. I doubt EY would have made the same arguments against Ron Paul.

The problem isn't that Trump is right but that he doesn't listen to experts, doesn't read and is a compulsive liar. Not the usual kind of lying that most politicians engage but also lying to the people around him. Trump got rich because he frequently didn't pay people what he owned them. The the campaign he suggested that he wants to do the same with US debt. It's bad moves that lead to bad foreign policy.

This is exactly the same as the argument for radical conservatism that the neoreactionaries make

No, it's a particular case of that argument which is much more plausible than almost all the others, because the realistic outcomes are so much more impactive. Reactionaries would have it that tinkering with gender norms brings on the apocalypse, too,

Upvoted. Thank you for hosting.

6 - Trump's move wasn't bad: Yudkwosky didn't take into account the support of other intellectuals in the same sphere

Both user:hg00 and I argued that Eliezer stopped searching for expert opinions in a motivated way, but I concluded that relying on expert opinion, which ultimately appears to indicate that Trump will probably have more negative effects on our foreign policy than Clinton, was correct anyway. The OP specifies that the purpose of the discussion is to evaluate methodology, and remains silent on the evaluation of conclusions. I request that the summary you've written be edited to reflect this. (Removing the phrase 'Trump's move wasn't bad' seems sufficient to me; maybe explicitly mention motivated cognition?)

I also tentatively suggest appending the summary to the OP once you expect that you won't have to edit it again.

Eliezer is being an idiot who forgot his own maxim to not rely on fictional evidence. This was a game. Let me repeat this: game.

It was a simulation. Simulations can be incredibly powerful educational tools, when they reflect reality. These sorts of simulations are meant to reflect reality (in as much as is possible in a 4-hour window).

I rather doubt its goals were to educate the players as to how the world really works.

That is entirely the point of these simulations. I've been in them before, and helped construct a simulation once. The entire point of the exercise is to educate the players as to how the world really works. The designers, who themselves have relevant real world experience, spend a massive amount of time obsessing over this.

The inversion of the usual ranking is weird.

It is and it was a distraction in getting to the core argument, for me at least.

High variance is a good thing if you're sleepwalking off a cliff already.

In the range of possible ways to describe the status quo, from 0/100 to 100/100, sleepwalking off a cliff is oddly specific and on one end of an extreme assessment.

Isn't the main argument that for most cases high variance is bad?

The inversion of the usual ranking is weird.

Sorry for that. But there could be a tower of meta-level, and if someone had invoked a level above level B, I wouldn't have known how to call it if it was named level A.

This was a game. Let me repeat this: game. I rather doubt its goals were to educate the players as to how the world really works.

On the other hand, one safe bet is that a game is always less complicated than the real thing is trying to simulate.
Without knowing the details, it's impossible to tell if the game was a sort of "big diplomacy game with ex diplomats just for fun" or a Dem publicity stunt. While possible, assuming it for certain is unwarranted.
Anyway, your argument would fall under the category: level B isn't real / doesn't matter.

Most random variations are disasters. Aren't we lucky that most variations are not random?

In the mind of Eliezer, as far as we know, he considers Trump pretty clueless (level B speaking), so that necessarily all his moves would be random.

Specifically, I don't think the average voter knows anything.

cf. Winston Churchill: "The best argument against democracy is a five-minute conversation with the average voter."

How can we trust the electorate to make these decisions?

You make an interesting distinction between "we" and "the electorate" X-D

Perhaps you should ask how could you trust yourself to make important decisions about your life.

I think these are important points, but an important counterpoint is the subject of The American System and Misleading Labels:

How much of the benefit of living in a democracy is in the small influences that voters occasionally manage to exert on the political process? And how much of that benefit is from power-wielders being too scared to act like historical kings and slaughter you on a whim?

Arguably, the chief historical improvements in living conditions have not been from voters having the influence to pass legislation which (they think) will benefit them, but, rather, from power-wielders becoming scared of doing anything too horrible to voters. Maybe one retrodiction (I haven't checked) would be that if you looked at the history of England, you would find a smooth improvement in living conditions corresponding to a gradually more plausible threat of revolution, rather than a sharp jump following the introduction of an elected legislature.

This debate is the main reason that I'm fascinated by post-democratic ideas, but dial my skepticism up to 11 with regards to their real-world consequences.

I have no idea if Trump or Clinton (or Gary Johnson) is "best" for America's interests

But then, if you have a better knowledge than the average voter and still couldn't decide who is better, what difference does it make?
If more knowledge is not able to influence your opinion on who to vote, then no harm is done by ignorance. It even saves you time to do other things.

Overall, the nature of the problem seems to be that our representative democratic institutions have been captured by meta-institutions that implement the iron law of oligarchy by systematically reducing the risk of change. They have done so by converging on a common set of policies that do not serve the public interest, but minimize the risk of the parties losing the corporate funding they require in order to achieve re-election. And in so doing, they have broken the "peaceful succession when enough people get pissed off" mechanism that prevents revolutions. If we're lucky, emergent radical parties will break the gridlock (here in the UK that would be the SNP in Scotland, possibly UKIP in England: in the USA it might be the new party that emerges if the rupture between the Republican realists like Karl Rove and the Tea Party radicals finally goes nuclear), but within a political generation (two election terms) it'll be back to oligarchy as usual."

It's a funny thing about Hillary which her campaign tried to downplay as much as they could, but she has no accomplishments. The standard "what are your major achievements?" question stumps her. She's successful at playing power games and pretty much nothing else -- starting from the health care reform under Clinton, to her tenure at State Dept, and ending with her presidential bid.

Some of Yudkowsky's arguments were good, but he was still an embarrassment to the movement. If I recall correctly he posted maybe half a dozen Facebook statuses to the effect of "OMG Trump is THE WORST" before offering any sort of argument. Of course, this plays in to the idea that people who oppose Trump are bullies who care more about optics than substance.

This style of argument seems unproductive.

The concrete accusation against Yudkowsky is apparently that he made several posts which mentioned his position on Trump before making the posts which laid out the reasoning behind his position. If that is a vice, it seems like a minor one.

It's possible that there was something specific about Yudkowsky's posts which made them worse, but there are no details given here, nor links which might allow someone to see what exactly he wrote and form their own opinion. Just an uncharitable paraphrase "OMG Trump is THE WORST". Uncharitable paraphrases are a fuel of unproductive political discussions, since they make it easy for people to talk past each other, get caught up in their preferred storylines, and collect reasons for disliking the other side.

The loaded terms (e.g., "embarrassment", "bullies") also aren't helpful if the goal is to have a conversation where people who disagree about Trump can talk with each other productively. They also don't seem to fit the behavior described. Yudkowsky thought that one of the two major party candidates would be a bad President, and he shared that opinion on Facebook several times before giving his reasoning. What a ?bully?!

If something about his posts was bullying, then call him out on it and be clear about what the bullying consisted of. If they weren't bullying and you're just concerned that other people might mistakenly see them as bullying and think less of the rationality community because of it, then talk about them in a way that makes it clear that they were not bullying. "Of course, this plays in to the idea that people who oppose Trump are bullies" does neither.

And certainly no signs of the Charles Darwin 'steelman' strategy. Did not make the best possible case for Trump before making the opposite case.

Politics does make being rational very difficult. He certainly got that one right.

Of course, this plays in to the idea that people who oppose Trump are bullies who care more about optics than substance.

These sources are very partisan and biased.

On 1, whilst that should reduce your belief that Trump himself will fire nuclear weapons, it is a fairly weak argument for several reasons: first, many many people have had something to protect yet engaged in reckless behaviour anyway with no regard for others. It's on a much smaller scale than starting a nuclear apocalypse but we should still consider that family members are only a weak protection, especially if, as others have argued, Trump is highly impulsive. Further, it only holds if Trump really does value his family highly, higher than his own self image, and given that he is a massive narcissist I'd say that's not guaranteed by any means.

The second issue is that there are more ways for MAD to occur than simply Trump ordering an attack. A nuclear war could be started by other nations, and there is reason to believe that Trump leads to a higher probability of this. First of all, he has encouraged nuclear proliferation, which will simply increase the number of actors capable of an attack and thus inevitably increasing the probability. Secondly, many non-nuclear nations may be tempted to acquire some if they believe they are undefended, which again is something that Trump has stated. Perhaps Japan feels endangered by North Korea and starts their own weapons program as the US pulls out troops or some such. I believe Scott Alexander made a similar argument over at SSC, probably better worded than mine.

I have no disagreements on point 2, and no comment on 4.

Regarding 3, I think this is a false argument. The issue I have is that 'Islam' is not a nation state or even a physical construct , you cannot start a nuclear war with 'Islam'. Presumably when you refer to Islam you are talking of the danger of a terrorist group or possibly ISIS. However, its difficult to see any MAD situation with such groups - for one, it is extremely unlikely that they are ever able to acquire more than one weapon and thus they can't 'retaliate' further. In addition, if they were able to succeed with a nuclear attack, there would also be nothing for the US or others to retaliate against. You cannot launch a nuclear strike against Al-Qaeda, and though ISIS do have land to attack they are now so spread out that it simply wouldn't make sense to go with a nuclear option over conventional warfare (albeit this can change over time).

So far, we have information on Trump's skill set as a businessman: immoral and unethical perhaps, but ultimately very successful.

He's gone bankrupt six times.

Question. I admit I have a low EQ here, but I"m not sure if 4) is sarcasm or not. It would certainly make a lot of sense if "I've been glad to see in this thread that we LW's do, in fact, put our money where our mouths are when it comes to trying to navigate, circumvent, or otherwise evade the Mindkiller." were sarcasm.

I would have said we had information on 2), but I've made so many wrong predictions about Donald Trump privately that I think my private opinion has lost all credibility there. 1) makes sense.

I can see why you might be afraid of war breaking out with Russia, but why do you consider Islam a major threat? Maybe you don't and I'm misinterpreting you, but given how little damage terrorist attacks actually do isn't Islam a regional problem to which the West has a major overreaction problem?

The example that springs to mind is the 19th century US. And what they did was conquer and drive off (or kill off) the existing residents west of the Mississippi to make room for the new immigrants.

How literally are we taking "uncontrolled"?

Trump hates Salafi/Saudi's ,

What makes you think that Trump hates the Saudi's? From what I have seen he decided against making a big deal about the fact that Hillary said in the Podesta emails that the Saudi's fund ISIS. Putting attention on the issue would have been good politics in the sense of raising his chances of getting elected. Not doing so, suggests that he doesn't want to alienate the Saudi's. If I missed news stories about Trump making that link, please provide links.

Trump suggested it's fine to protect Saudi's if they pay. The Saudi's do pay for the weapons that get send to them.

His comments about the Saudi's getting nukes are problematic. Someone who wants to oppose the Saudi's should consider it important that they don't have nukes.

The situation in Syria is too important you guys, nothing else matters if we bring that one home we get our prestige back

I don't think that the world would give a president Trump prestige even if he ends the Syrian war.

I would like this to be a comment on methodology, about if their arguments were sound given what they knew and believed. I most definitely do not want this to decay in a lamentation about the results

So... any comments on methodology?