[LINK] Another "LessWrongers are crazy" article - this time on Slate

I want this list posted in response to every "is there anything we should do" ever. Just all over the internet. I would give you more than one upvote just for that list if I could.

This is also such a delicious example of the Streisand effect...

Yes, Eliezer's Streisanding is almost suspiciously delicious. One begins to wonder if he is in thrall to... well, perhaps it is best not to speculate here, lest we feed the Adversary.

I think this is also a delicious example of how easy it is to troll LessWrong readers. Do you want to have an LW article and a debate about you? Post an article about how LW is a cult or about Roko's basilisk. Success 100% guaranteed.

Think about the incentives this gives to people who make their money by displaying ads on their websites. The only way we could motivate them more would be to pay them directly for posting that shit.

This is also such a delicious example of the Streisand effect...

I was going to say precisely that. In the end, banning Roko's post was pointless and ineffectual: anyone with internet access can learn about the "dangerous" idea. Furthermore, it's still being debated here, in a LessWrong thread. Is any of you having nightmares yet?

If an AI article is used, I nominate sorting pebbles into correct heaps.

Let's check featured articles on the main page on 19 July 2014....and...there we go.

I like this idea. Of course, right now the top thing on discussion is this thread, so here is probably as good a place as any to link the good stuff. One of my own personal favorites: Yvain's Diseased Thinking. Or, more or less the same point in much more concise form: Eliezer's Disguised Queries.

Also, the top-ranked posts page, which I'm not sure is linked anywhere obvious.

Since LW is going to get a lot of visitors someone should put an old post that would make an excellent first impression in a prominent position. I nominate How to Be Happy.

The problem isn't that easy to solve. Consider that MIRI, then SIAI, already had a bad name before Roko's post, and before I ever voiced any criticism. Consider this video from an actual AI conference, from March 2010, a few months before Roko's post. Someone in the audience makes the following statement:

Whenever I hear the Singularity Institute talk I feel like they are a bunch of religious nutters...

Or consider the following comment by Ben Goertzel from 2004:

Anyway, I must say, this display of egomania and unpleasantness on the part of SIAI folks makes me quite glad that SIAI doesn’t actually have a viable approach to creating AGI (so far, anyway…).

And this is Yudkowsky's reply:

[...] Striving toward total rationality and total altruism comes easily to me. [...] I’ll try not to be an arrogant bastard, but I’m definitely arrogant. I’m incredibly brilliant and yes, I’m proud of it, and what’s more, I enjoy showing off and bragging about it. I don’t know if that’s who I aspire to be, but it’s surely who I am. I don’t demand that everyone acknowledge my incredible brilliance, but I’m not going to cut against the grain of my nature, either. The next time someone incredulously asks, “You think you’re so smart, huh?” I’m going to answer, “Hell yes, and I am pursuing a task appropriate to my talents.” If anyone thinks that a Friendly AI can be created by a moderately bright researcher, they have rocks in their head. This is a job for what I can only call Eliezer-class intelligence.

LessWrong would have to somehow distance itself from MIRI and Eliezer Yudkowsky.

The Useful Idea of Truth

Uh, no. Surprisingly few "rich dudes" have shown an interest in cryonics. I know quite a few cryonicists and I have helped to organize cryonics-themed conferences, and to the best of my knowledge no one on the Forbes 500 list has signed up.

Peter Thiel seems to be on the Forbes 500 list. Are you arguing that he isn't signed up for cryonics? Are you saying he simply isn't attending those conferences?

the things rich people really care about tend to become status signals and attract adventuresses in search of rich husbands.

Can you give examples?

Ordinary people seem be in thehabit of frittering their life insurance away in their descendants. Perhaps cryoni.cs is for the moderately well off and single.

To be fair, the article got lots of things wrong.

If Langford basilisks actually existed, Gawker would be the first site to use them.

I find belief in basilisks ridiculous. Arguing that an idea could do harm by merely occupying space in a brain is a tremendous discredit to humanity. Any adult brain that is so vulnerable as to suffer actual emotional damage by the mere contemplation of an idea is a brain accustomed to refusing to deal with reality. If the fact is that The Basilisk wants to torture me, I want to believe that it wants to torture me.

From a technical standpoint, this bit:

Even if the alien jeers at you, saying, “The computer said you’d take both boxes, so I left Box B empty! Nyah nyah!” and then opens Box B and shows you that it’s empty, you should still only take Box B and get bupkis. ... The rationale for this eludes easy summary, but the simplest argument is that you might be in the computer’s simulation. In order to make its prediction, the computer would have to simulate the universe itself.

Seems wrong. Omega wouldn't necessarily have to simulate the universe, although that's one option. If it did simulate the universe, showing sim-you an empty box B doesn't tell it much about whether real-you will take box B when you haven't seen that it's empty.

(Not an expert, and I haven't read Good and Real which this is supposedly from, but I do expect to understand this better than a Slate columnist.)

And I think the final two paragraphs go beyond "pokes fun at lesswrong".

Out of curiosity, what is the fraction of LW posters that believes this is a genuine risk?

Vanishingly small - the post was deleted by Eliezer (was that what, a year ago? two?) because it gave some people he knew nightmares, but I don't remember anybody actually complaining about it. Most of the ensuing drama was about whether Eliezer was right in deleting it. The whole thing has been a waste of everybody's time and attention (as community drama over moderation almost always is).

If only Less Wrong members had the common sense of the average Salon reader, then they would instantly see through such silly dilemmas.

Not sure I agree with your point. There's a standard LW idea that smart people can believe in crazy things due to their environment. For "environment" you can substitute "non-LW" or "LW" as you wish.

Out of curiosity, what is the fraction of LW posters that believes this is a genuine risk?

From what I've seen, it seems like very few people who know the basilisk believe it (<10 maybe?), but there are some people (still not a lot, but significantly more than 10), who avoid the basilisk just in case it is dangerous, because of EY's reaction.

I wonder why it's so hard to get positive coverage for LW and so easy to get negative coverage, when in contrast Wired magazine gave Cypherpunks a highly positive cover story in 1993, when Cypherpunks just got started and hadn't done much yet except publish a few manifestos.

There's an easy answer and a hard answer.

The easy answer is that, for whatever reason, the media today is far more likely to run a negative story about the tech industry or associated demographics than to run a positive story about it. LW is close enough to the tech industry, and its assumed/stereotyped demographic pattern is close enough to that of the tech industry, that attacking it is a way to attack the tech industry.

Observe:

"highly analytical sorts interested in optimizing their thinking, their lives, and the world through mathematics and rationality ... techno-futurism ... high-profile techies like Peter Thiel ... some very influential and wealthy scientists and techies believe it ... computing power ... computers ... computer ... mathematical geniuses Stanislaw Ulam and John von Neumann ... The ever accelerating progress of technology ... Futurists like science-fiction writer Vernor Vinge and engineer/author Kurzweil ... exponential increases in computing power ... Yudkowsky and Peter Thiel have enthused about cryonics, the perennial favorite of rich dudes ... the machine equivalent of God ... rational action ... a smattering of parallel universes and quantum mechanics on the side ... supercomputer ... supercomputer ... supercomputer ... supercomputer ... autism ... Yudkowsky and other so-called transhumanists are attracting so much prestige and money for their projects, primarily from rich techies ... messianic ambitions, being convinced of your own infallibility, and a lot of cash"

Out of the many possible ways to frame the article, Slate chose to make it about "rich techies". Why postulate that Omega has a supercomputer? Why repeat the word 'supercomputer' four times in five sentences? The LW wiki doesn't mention computers of any sort, and the Wikipedia article only uses the word 'computer' twice. advancedatheist said above that the cryonics claim is false; assuming he's right, why include a lie?

It's clearly not a neutral explanation of the Basilisk -- and it fits into a pattern.

The hard answer would include an explanation of this pattern. (I'm not sure whether it would be a good idea to speculate about this in this particular thread, so: anyone who's tempted to do so, take five minutes and think over the wisdom of it beforehand.)

Personal contact via the people employed in the Wired magazine and a lot of hackers are quite strong. Wired had actually an intention of pushing projects like Cypherpunks or in the last years the Quantified Self movement which they essentially founded (Keven Kelly and Gary Wolf are both Wired Editors).

I don't think that LW is really the place that needs positive PR. I can't really think of a story about LW that I want to tell a reporter. I can think of stories about MIRI or about CFAR but LW itself doesn't need PR.

Well, for Charlie Stross it's practically professional interest :P

Anyhow: anecdote. Met an engineer on the train the other day. He asked me what I was reading on my computer, I said LW, he said he'd heard some vaguely positive things, I sent him a link to one of Yvain's posts, he liked it.

"Doesn't work against a perfectly rational, informed agent" does not preclude "works quite well against naïve, stupid newbie LW'ers that haven't properly digested the sequences."

Memetic hazard is not a fancy word for coverup. It means that the average person accessing the information is likely to reach dangerous conclusions. That says more about the average of humanity than the information itself.

There were several possible fairly-good reasons for deleting that post, and also fairly good reasons for giving Eliezer some discretion as to what kind of stuff he can ban. Going over those reasons (again) is probably a waste of everybody's times. Who cares about whether a decision taken years ago was sensible, or slightly-wrong-but-within-reason, or wrong-but-only-in-hindsight, etc. ?

Also, in Newcomb's problem, the goal is to go away with as much money as possible. So it's obvious what to optimize for.

What exactly is the goal with the Basilisk? To give as much money as possible, just to build an evil machine which would torture you unless you gave it as much money as possible, but luckily you did, so you kinda... "win"? You and your five friends are the selected ones who will get the enjoyment of watching the rest of humanity tortured forever? (Sounds like how some early Christians imagined Heaven. Only the few most virtuous ones will get saved, and watching the suffering of the damned in Hell will increase their joy of their own salvation.)

Completely ignoring the problem that just throwing a lot of money around doesn't solve the problem of creating a safe recursively self-improving superhuman AI. (Quoting Sequences: "There's a fellow currently on the AI list who goes around saying that AI will cost a quadrillion dollars—we can't get AI without spending a quadrillion dollars, but we could get AI at any time by spending a quadrillion dollars.") So these guys working on this evil machine... hungry, living in horrible conditions, never having a vacation or going on a date, never seeing a doctor, probably having mental breakdowns all the time; because they are writing the code that would torture them if they did any of that... is this the team we could trust with doing sane and good decisions, and getting all the math right? If no, then we are pretty much fucked regardless of whether we donated to the Basilisk or not, because soon we are all getting transformed to paperclips anyway; the only difference is that 99.9999999% of us will get tortured before that.

How about, you know, just not building the whole monster at the first place? Uhm... could the solution to this horrible problem really be so easy?

Yes, the Basilisk does address how one should act in real life. It says: 'Don't build a basilisk, dummy!'. Problem solved.

Newcomb's problem applies just as much in real life. I could in real life appraoch you claiming to be Omega, or as an agent of the Matrix or whatever. How would you respond?

If you generalize to "acausal trades" (suspect, IMHO, but commonly accepted here), you get Roko's basilisk. A proper decision theory would advise you not to negotiate with terrorists, so Roko's basilisk is not a real concern. But honestly the subset of people that really understand decision theory enough to find that out on their own is quite small, hence the ban.