How to improve the public perception of the SIAI and LW?

(1) Write a short, introductory, thoroughly cited guide on each major concept employed by SIAI / LW.

As an example, this is what I'm currently doing for the point about why standard, simple designs for machine ethics will result in disaster if implemented in a superintelligent machine. Right now, you have to read hundreds of pages of dense material that references unusual terms described in hundreds of other pages all across Less Wrong and SIAI's website. That is unnecessary, and doesn't help public perception of SIAI / LW. It looks like we're being purposely obscurantist and cult-like.

Why an intelligence explosion is probable is another good example of this.

(2) Engage the professional community. Somebody goes to SIAI's page and looks for accomplishments and they see not a single article in a peer-reviewed journal. Compare this to, um... the accomplishments page of every other 10-year research institute or university research program on the planet.

EDIT: I should note that in the course of not publishing papers in journals and engaging the mainstream community, SIAI has managed to be almost a decade ahead of everyone else. Having just read quite nearly the entirety of extant literature in the field of machine ethics, I can say with some confidence that the machine ethics field still isn't caught up to where Eliezer was circa 2001.

So of course SIAI can work much more quickly if it doesn't bother to absorb the entirety of the (mostly useless) machine ethics literature and then write papers that use the same language and style as the mainstream community, and cites all the same papers.

The problem is that if you don't write all those papers, then people keep asking you dumb questions about "Why can't we just tell it to maximize human happiness?" You have to keep answering that question because there is no readable, thoroughly-cited, mainstream-language guide that answers those types of questions. (Except, the one I'm writing now.)

Also, not publishing those papers in mainstream journals leaves you with less credibility in the eyes of journals who are savvy enough to know there is a difference between conference papers and those accepted to mainstream journals.

So I think it's worth all that effort, though probably not for somebody like Yudkowsky. He should be working on TDT and CEV, I imagine. Not reading papers about Kantian solutions to machine ethics.

Perhaps it would be useful to change the framing?

For example... if I join a book discussion group:

• I understand that much of the discussion will not make any sense to me if I haven't read the book, and that there's a rapidly reached limit to how usefully I can participate in the discussion without having read the book.

• I don't expect anyone to expend a lot of effort justifying the reading of the book to people skeptical about the benefits of doing so.

• I don't expect anyone to summarize just the interesting parts of the book for me so I can participate in the discussion without actually reading the book.

All of this remains true even if the group welcomes new members who haven't read the book yet, but who hang around because the community's discussions seem interesting.

So, perhaps encouraging a similar attitude with respect to the Sequences would help manage some of the PR issues you identify surrounding them.

Of course, none of that would address the SIAI-related issues. Then again, from my perspective LW is already fairly separate from SIAI... at least, I participate in the former and not in the latter and nobody seems to mind... so I don't see a problem that needs solving there.

But I would not object to further separation, if a consensus emerged in favor of that.

First one has to identify characteristics that could potentially be used to cast a damaging light on this community. Here the most obvious possibility seems to be to portray the SIAI, together with LessWrong, as a cult.

Probably the other main possibilities that spring to my mind are:

• That it is a luddite organisation;

• That it is an unscrupulous machine intelligence outfit masquerading as a luddite organisation for marketing reasons;

• That it has fallen too far behind to have much chance of meeting its goals;

• That it is too perfectionist to have much chance of meeting its goals;

• That its lack of experience and secretive culture are a bad combination.

Less Wrong has a FAQ that anyone can edit. I think your first four Measures could be best addressed by getting an account on the Wiki and writing what you think we need.

Your worries about the structure of SIAI sound like the sort of thing worth talking about, but posting them on Less Wrong might not be the best way to go about them due to bystander effect and lots of important SIAI folk not being readers here. If you are really interested in this side of things, consider emailing someone on the organizational side of SIAI (Eliezer is not primarily organizational side and is usually busy; Michael Vassar might be good or at least know who to forward it to) and seeing what they have to say. Justin Shovelain also has a history of being good at explaining this side of things; he has a sequence that I think will get some of this across somewhere in the pipeline.

Using scope insensitivity and high-risk to justify action, outweigh low probabilities and disregard opposing evidence.

This is true, but I think the problem goes even further than this: many people are unwilling to make what Less Wrong readers would consider the "obvious" utilitarian choice, e.g. in a scenario like Torture vs. Dust Specks. Outsiders probably consider these unintuitive moral decisions weird at best and scary at worst.

.

Advertising for LessWrong is plausible. I just got a £75 Google AdWords voucher in the post ...

(£75 is approximately nothing - a decent taster campaign is at least £300. But I have no use for it and LW is welcome to it ...)

You talk about "making people read the sequences". I suggest that "making" people to do anything doesn't work. You have to pull them. This means you need them to think there's something good there they want to have.

(You want to herd cats, you need to work out the local value of tuna.)

How about some advertising taglines? The current tagline is excellent, for example. But why would people want that? What can they get here they don't have now?

People want to WIN.

Most people don't feel like winners.

• "Win in the world with clear, rational thinking"
• "If you know why you do things, you can WIN in the world."

etc. Any others? Other ideas of things that will pull people towards LW?

Edit: And why has HP:MoR lured people in? What kept them here? How many came here from HP:MoR and did not stay? Why not? Etc.

Discourage the downvoting of people who have not yet read the Sequences.

I don't downvote based on whether people have read the sequences. I vote based on merit and obnoxiousness.

Occasionally people bother taking LW seriously enough to criticise, e.g. this FWIW. I suggest this is a useful sort of notice and that LW could do with a lot more of it.

Detach LessWrong from the SIAI by creating an additional platform to talk about related issues.

I think LessWrong is sufficiently seperated from SIAI - most LessWrong members are not involved at all with SIAI, and many SIAI members/advisors/whatchamacallits don't post on LessWrong (I think).

I think of LessWrong more of as "a place for people who read the Sequences to discuss related topics", whereas SIAI is much more focused on a specific goal. SIAI people may make anouncements on LessWrong because a lot of members are interested, but I don't expect SIAI people to pay that much attention to LessWrong in general.

So in that light, these:

Ask or pay independent experts to peer-review. Make the finances of the SIAI easily accessible. Openly explain why and for what the SIAI currently needs more money.

... are for SIAI, and have little to do with LessWrong.