Short Primers on Crucial Topics

Thanks for laying out these specific proposals, Luke.

Whatever you guys decide to go with, I am much more tempted to donate to SI because of these.

The AI Risk wiki still seems worth doing first, because it increases the proportion of Luke Muehlhausers to Holden Karnofskys it creates by rounding out the details of the argument.

Without doing any cost-benefit analysis, I can tell you that, of the three so far, this one gives me by far the most fuzzies, just thinking about it. A scholarly wiki? Boring. Research? Boring. Short primers on crucial topics??? That sounded less boring in my head.

I couldn't tell you why this happened. Maybe I just really liked Facing the Singularity more than I realized. Does anyone else have a similar reaction?

I like this idea overall, but SI might want to think twice before it risks becoming known as a group that leaves books promoting its ideas lying around.

Also, I'm doubtful of the claim that such introductory books can only be usefully produced by top Less Wrong contributors.

Unlike the wiki idea, this is something that I can wholeheartedly endorse. Even shorter summaries of the primer (flyer or one-page sizes) would be good too.

$1000 on book design, Kindle version production, etc.

I would recommend at least doubling this budget, I think (with the understanding that you don't have to spend it all). These should look really appealing, and it might be beneficial for them to be illustrated on the interior.

Is the cover design shown here (1) just for fun here on LW, or (2) something you're thinking of actually doing on actual kinda-book-like entities?

If the latter, then you might want to reconsider the merits of making it quite so blatant a ripoff of the famous "Very Short Introduction" series of books. That seems like it might ring some readers' confidence-trick alarm bells. (It certainly does mine.)

Looking at the page of Facing the Singularity I just realized again how wrong it is from the perspective of convincing people who are not already inclined to believe that stuff. The header, the title, the text...wrong, wrong, wrong!

Facing the Singularity

The advent of an advanced optimization process and its global consequences

Sometime this century, machines will surpass human levels of intelligence and ability. This event — the “Singularity” — will be the most important event in our history, and navigating it wisely will be the most important thing we can ever do.

The speed of technological progress suggests a non-negligible probability of the invention of advanced general purpose optimization processes, sometime this century, exhibiting many features of general intelligence as envisioned by the proponents of strong AI (artificial intelligence that matches or exceeds human intelligence) while lacking other important characteristics.

This paper will give a rough overview of 1) the expected power of such optimization processes 2) the lack of important characteristics intuitively associated with intelligent agents, like the consideration of human values in optimizing the environment 3) associated negative consequences and their expected scale 4) the importance of research in preparation of such a possibility 5) a bibliography of advanced supplementary material.

...say, at the top math, computer science, and formal philosophy departments in the English-speaking world.

People at top academic departments everywhere in the world speak English... (which is probably true even for the janitor when it comes to some western countries).

A guy who works for a book publisher once told me that they pay about 8 euro's per 1000 words to a good translator for books they translate from foreign languages. So by this calculation you can have a 100.000 words text translated in Romanian for 800 euros.

The wiki, research and shorter primers proposals signals a little strategic turn in SI. To me, the shorter primer proposal sounds more academic, even if they not come with the "Oxford University Press" in the front.

Before you continue with this you should maybe try to get someone important read 'Facing the Singularity' without trying too hard. If that doesn't work then...

I have my doubts that someone like Terence Tao would read your primer.

For some time now I am watching the real-time stats for my homepage, especially when I post links at places where people of similar calibre to Terence Tao are chatting. And I seldom get more than 2 clicks, even if more than 20 people converse in that thread.

Now it is true that I am a nobody, why would they read a post written on my personal blog? But how would they know that something called 'Facing the Singularity' is more worthy of their attention?

If I really wanted to I would probably be able to get them read my stuff. But that's difficult and would probably take a middleman who shares a link to it on his blog/Google+/Facebook page and whose stuff is subsequently read by top-notch people.