2017 LessWrong Survey

post by namespace (ingres) · 2017-09-13T06:26:42.893Z · LW · GW · Legacy · 75 comments

The 2017 LessWrong Survey is here! This year we're interested in community response to the LessWrong 2.0 initiative. I've also gone through and fixed as many bugs as I could find reported on the last survey, and reintroduced items that were missing from the 2016 edition. Furthermore new items have been introduced in multiple sections and some cut in others to make room. You can now export your survey results after finishing by choosing the 'print my results' option on the page displayed after submission. The survey will run from today until the 15th of October.

You can take the survey below, thanks for your time. (It's back in single page format, please allow some seconds for it to load):

Click here to take the survey

75 comments

Comments sorted by top scores.

comment by pepe_prime · 2017-09-13T13:20:21.241Z · LW(p) · GW(p)

[Survey Taken Thread]

By ancient tradition, if you take the survey you may comment saying you have done so here, and people will upvote you and you will get karma.

Let's make these comments a reply to this post. That way we continue the tradition, but keep the discussion a bit cleaner.

Replies from: pepe_prime, Elo, enye-word, Screwtape, Lark, g_pepper, None, RowanE, ingres, gbear605, adamzerner, NancyLebovitz, Zvi, SoerenE, JenniferRM, maxjmartin, LawChan, Bitnotri, RainbowSpacedancer, DragonGod, Regex, richard_reitz, moridinamael, Gyrodiot, eukaryote, gjm, Artaxerxes, J_Thomas_Moros, None, blob, iceman, scarcegreengrass, fortyeridania, EricHerboso, tenthkrige, Viliam, plethora, jkadlubo, marchdown, alexey, kvas_duplicate0.1636121129676118
comment by pepe_prime · 2017-09-13T13:20:36.667Z · LW(p) · GW(p)

I have taken the survey.

comment by Elo · 2017-09-13T13:27:19.173Z · LW(p) · GW(p)

I took the survey. Towards a better future!

comment by enye-word · 2017-09-14T05:33:34.707Z · LW(p) · GW(p)

I have also taken this survey.

comment by Screwtape · 2017-09-13T15:13:36.221Z · LW(p) · GW(p)

I have taken the survey. I don't remember there being a public key question at the end of the last one though, which is a shame since that obviously means I don't remember what mine was last time.

Replies from: ingres
comment by namespace (ingres) · 2017-09-13T19:00:55.324Z · LW(p) · GW(p)

There wasn't. There was one on the 2014 survey. It's one of the missing items I decided to add back in.

comment by Lark · 2017-09-14T00:54:45.432Z · LW(p) · GW(p)

Done!

comment by g_pepper · 2017-09-13T22:30:36.561Z · LW(p) · GW(p)

I have taken the survey.

comment by [deleted] · 2017-09-13T21:06:53.526Z · LW(p) · GW(p)

I took the survey!

comment by RowanE · 2017-09-13T19:00:52.205Z · LW(p) · GW(p)

I have taken the survey.

comment by namespace (ingres) · 2017-09-13T18:52:39.067Z · LW(p) · GW(p)

I have taken the survey.

comment by gbear605 · 2017-09-13T18:36:31.761Z · LW(p) · GW(p)

I took the survey. I feel like the questions that ask for numeric answers about the probability of AI risk should have been optional because I have very weak fews about them

Replies from: ingres
comment by namespace (ingres) · 2017-09-13T18:51:35.140Z · LW(p) · GW(p)

Every question is optional besides the first two.

comment by NancyLebovitz · 2017-09-13T16:32:29.297Z · LW(p) · GW(p)

I have taken the survey.

comment by Zvi · 2017-09-14T17:11:48.836Z · LW(p) · GW(p)

I have taken the survey.

comment by SoerenE · 2017-09-14T13:08:51.974Z · LW(p) · GW(p)

I have taken the survey.

comment by JenniferRM · 2017-09-14T10:40:31.016Z · LW(p) · GW(p)

I took the survey and upvoted every comment already here.

comment by maxjmartin · 2017-09-14T09:44:50.745Z · LW(p) · GW(p)

Survey completed

comment by LawrenceC (LawChan) · 2017-09-14T00:41:03.787Z · LW(p) · GW(p)

I took the survey!

comment by Bitnotri · 2017-09-14T17:03:18.830Z · LW(p) · GW(p)

Taken

comment by RainbowSpacedancer · 2017-09-14T15:34:57.335Z · LW(p) · GW(p)

I have taken the survey.

comment by DragonGod · 2017-09-15T14:34:13.675Z · LW(p) · GW(p)

I took the survey, phew that was long. I added a public key for what it's worth.

comment by Regex · 2017-09-15T05:22:47.797Z · LW(p) · GW(p)

I have taken the survey... away from everyone.

No one can have it.

It lives under my bed now.

comment by richard_reitz · 2017-09-15T04:13:03.852Z · LW(p) · GW(p)

I have taken the survey.

comment by moridinamael · 2017-09-14T20:41:49.730Z · LW(p) · GW(p)

It is done.

comment by Gyrodiot · 2017-09-14T15:54:12.755Z · LW(p) · GW(p)

I have taken the survey. Cannot remember my old public key, shrug

comment by eukaryote · 2017-09-17T07:46:04.680Z · LW(p) · GW(p)

I have taken the survey, please shower me in karma.

comment by gjm · 2017-09-15T15:06:38.550Z · LW(p) · GW(p)

I either have taken the survey in the last couple of hours, or have begun taking it and hereby commit to submitting a completed survey in the next couple of hours.

(Expressed in this form so as not to leak information about which survey is whose.)

[EDITED to add:] This is just to confirm that I have in fact completed the survey. I have also upvoted all the other taken-the-survey comments currently present.

comment by Artaxerxes · 2017-09-15T13:13:55.805Z · LW(p) · GW(p)

I did it!

comment by J Thomas Moros (J_Thomas_Moros) · 2017-09-17T04:26:32.103Z · LW(p) · GW(p)

I have completed the survey and upvoted everyone else on this thread

comment by [deleted] · 2017-09-15T23:18:52.536Z · LW(p) · GW(p)

I have taken the survey!

comment by blob · 2017-09-16T12:42:55.454Z · LW(p) · GW(p)

I've taken the survey.

comment by iceman · 2017-09-17T21:05:29.113Z · LW(p) · GW(p)

Survey taken!

comment by scarcegreengrass · 2017-09-19T15:59:44.391Z · LW(p) · GW(p)

I took the survey. It's probably my favorite survey of each year :) Thanks.

comment by fortyeridania · 2017-09-18T05:37:45.891Z · LW(p) · GW(p)

Done.

comment by EricHerboso · 2017-09-18T03:52:32.032Z · LW(p) · GW(p)

I have taken the survey.

comment by tenthkrige · 2017-09-20T13:55:40.936Z · LW(p) · GW(p)

I have also taken the survey.

comment by Viliam · 2017-09-19T21:47:00.785Z · LW(p) · GW(p)

Done.

comment by plethora · 2017-09-21T09:54:16.291Z · LW(p) · GW(p)

I have taken the survey.

comment by jkadlubo · 2017-09-21T08:24:30.826Z · LW(p) · GW(p)

I've taken the survey. Possibly my first activity here this year

comment by marchdown · 2017-09-23T02:38:49.994Z · LW(p) · GW(p)

Survey taken.

comment by alexey · 2017-09-22T20:34:26.837Z · LW(p) · GW(p)

I've taken the survey.

comment by kvas_duplicate0.1636121129676118 · 2017-09-21T12:35:04.917Z · LW(p) · GW(p)

I took the survey. It was long but fun. Thanks for the work you've put into designing it and processing the results.

comment by Aharon · 2017-09-13T11:44:08.040Z · LW(p) · GW(p)

Thanks for your work in creating the survey, and for LesserWrong. I shared the link to the survey in our meet-up group, and hope many people will contribute.

comment by gjm · 2017-09-15T15:39:37.773Z · LW(p) · GW(p)

A few comments:

Q10: misspelling: "monogomous" should be "monogamous"

Q27: not clear what "as a community initiative" means. My actual impression is that LW2.0 is a project a few people are working on, but that most of "the community" has little visibility of it or input into it.

Q31: really needs an "insufficient data for meaningful answer" option.

Q32: fails to distinguish between "this is an incredibly important goal" and "this is the only goal that matters". If I became 100% convinced that no version of LW was ever going to raise the general sanity waterline I would be disappointed (note: I am already somewhat disappointed) but I wouldn't leave because I like being part of the community for other reasons too.

Q33: totally unclear what it's asking, and it seems like there's far too little information to give an answer that means anything. What is "this forum"? Specifically cafechesscourt (about which I know nothing), or is the picture just there to add colour? The question seems to presuppose that I can tell whether a forum would be worth participating in given only the identity of the person running it, which is bonkers.

Q33ff: there seems to be an underlying assumption here that what LW primarily needs is a technically better forum. I think the problems LW has faced have mostly been social rather than technical, and the main impact technical problems have had is that they have made it harder to address some social issues. (For instance, it was difficult to react appropriately to sustained malfeasance by a political fanatic because the LW server was (a) under the control of people who had little stake in its success and (b) running code that was painful to work with. But the actual problem was the political fanatic's behaviour.)

Q36: shouldn't forbid comments on unchosen options. Don't you want to know why someone doesn't care about something that sounds like it might matter?

Mental health questions: Might be worth distinguishing between "confident I would get a formal diagnosis if examined" and "think I probably have something similar". (The depression question 51, if you pick either version of "yes", opens up a question 52 asking whether you "still qualify for a depression diagnosis". If that distinction doesn't matter, maybe it should say something more like "still suffer from depression". Incidentally, I don't know why that question does this but the others don't. I guess depression more often goes away completely than the others.)

Q67: for some of these, some people with sufficient natural aptitude might have attained a high level of skill without the extreme effort implied by the rightmost column. (E.g., I'm a professional mathematician; I never "practised calculus every day" so far as I can recall.) I am guessing that in such cases you still want that rightmost column ticked (and maybe I should consider effort on more difficult branches of mathematics related to calculus to qualify) but it's not entirely clear. It's also unclear how broadly to take some of the categories; e.g., if someone has put a lot of effort into musical performance, does that count as "painting/drawing/etc."? (Depends whether "etc." is meant to cover other artistic endeavours or other forms of visual art or what.)

Q71,72: I wonder whether you want one more question, asking to what extent people think research into friendly AI is important. (So then you have a three-level funnel: worth working on friendly AI? worth doing it mathematically? is MIRI doing that well? rather than a two-level funnel.)

Q92: is kinda meaningless without knowing what country the respondent is from. (At least in so far as their opinion on this is an actual opinion rather than a mere party label.)

Q100ff: I think the biggest barriers to feasibility of replacing humans in many occupations, even discounting cases where being nonhuman is as such an obstacle, will likely not be down to intelligence unless that's broadened a great deal. Consider, e.g., opera singers or prostitutes. Machines trying to do these jobs will need to look and sound human not only because of people saying "ewwww, fake person" but also because the jobs intrinsically require specifically human-like performance.

Q108: crying out for a probability estimate rather than yes/no. I suspect most people signed up for cryonics think it probably won't work.

Q115ff: unclear (and I think it matters a lot for many people's opinions and feelings) what sort of modification is envisaged and how reliably it's supposed to be known to work. E.g., I have a tweenaged child and if someone said "here is a thing we can do that will increase her intelligence" then, quite apart from the fact that it would need to be her decision as well as mine, I would be incredibly skeptical and greatly disinclined to try the treatment for fear of unforeseen adverse consequences. And even if it were somehow known to be safe, not to mess up people's personalities, etc., it would feel like a big scary intervention in an existing person's life, whereas an in utero treatment might feel quite different. I would guess that this sort of thing will only (at least in the foreseeable future) be feasible in utero, or perhaps even in vitro before implantation, but the question doesn't make it clear what hypothetical we're supposed to be entertaining. (And the reference to "your child" makes it hard for those who have children not to think in terms of modifications to the children they already have, however old.)

Q122: really needs a "don't know / no strong opinion / it varies" option.

Q124: really needs probabilities or something. I can totally envisage scenarios where "the end of work" is a big win and others where it's a huge disaster.

Q127: really wants to be accompanied by some sort of estimate of how probable the event is. If someone thinks there's a 10^-6 chance of any of these things, it doesn't matter much which they think most likely.

Replies from: ingres
comment by namespace (ingres) · 2017-09-15T20:37:27.989Z · LW(p) · GW(p)

Thanks for your feedback. While I agree with you at many (even most) points, there are several considerations to keep in mind:

  • It is not possible for me to change the questions at this point in time, There are nearly 200 responses at this point and it would be completely unfair of me to force everyone to retake the survey so I can fix most of the offered feedback. As a consequence I can only take these as potential improvements for the next survey.

  • The questions in the AI Progress section are ripped directly from an associated study, I have no control over their content or methodology besides replicating them as accurately and faithfully as I can.

  • Some simplification of scenarios is necessary to make them fit into a <150 question survey. There are also real limits to how much effort I can expect from people in terms of engaging with a scenario and that is why certain things such as the probability that a genetic treatment will be safe aren't included. Many things could use a probability but if I asked for one every time people would probably get frustrated with the survey and give up.

In the future we're looking at changing platforms so that the survey can be offered in a 'module' format which allows it to be taken in chunks over a much longer period of time with more detail in individual sections.

  • Quite often it is useful for a survey to explicitly not include a neutral option. Picking between two things is mentally difficult and a neutral option offers a path of least resistance which people are more likely to click than put in the effort even if that doesn't represent their true opinion.

  • At analysis time I have access to a respondents previous answers.

  • Our survey software prevents me from doing certain things which would be desirable.

Responding to certain specific questions:

Q33: Yes, CafeChesscourt. The question is "given nothing but the appearance of this forum" (CafeChesscourt put approximately zero effort into software, making it a useful control about how important software is to a forums success), would you use it if one of these "celebrity users" listed below were someone that ran it and used it regularly? The purpose of this question is to gauge how useful it is to get people who are popular to endorse a discussion forum. And while you might think it's bonkers to choose a forum that way, empirically many people will show up to places that are often reviled like Tumblr if the right person asks them to.

Q33ff: I agree that is an assumption many people are making in regards to this issue, that assumption is not there in the question however as its purpose was me teasing at the idea that perhaps focusing on technical excellence is the wrong metric.

Q36: I believe this is a limitation of the software, though I'll go ahead and double check since letting people write stuff in wouldn't materially effect the survey results.

Q52: Depression does go away more often than the others. But more importantly on the 2016 survey we had somewhat horrifying rates of depression. It was difficult to distinguish if this was because everybody gets depression at least once and then 'gets over it' or because that many LessWrongers are actually depressed. This question lets us tell the difference.

Q67: Yes. As you can tell this is a hard question to ask people but I think it's important enough to be worth asking even if imperfectly. In a future survey Calculus could probably be changed to "Calculus or 'higher maths'". The painting/etc option is meant to apply to visual arts. In the future it would probably be better to more accurately specify. In the grand scheme of things it will probably not spoil this years survey results to have the occasional musician pick it under the impression it counts. (I should also add music, thanks for the tip.)

Update (Fri Sep 15 13:46:59 PDT 2017): The issue in question 36 turned out to be fixable, thanks for the help!

Replies from: gjm
comment by gjm · 2017-09-18T08:33:38.488Z · LW(p) · GW(p)

For the avoidance of doubt, I appreciate your "several considerations" and in particular was not suggesting that you should hack the questions about mid-survey. And, er, I realise that I just posted a bunch of criticism without adding: thank you very much for doing the survey; I think it will be interesting and useful; the fact that I have some quibbles doesn't make that any less true. So please consider that added :-).

As a single data point, on Q33 I attempted to answer as if the question meant something like "If all you knew about a new forum was that X was running the show, would you be likely to check it out for that reason?" on the grounds that that was the most non-bonkers interpretation I could give the question. If it was meant to be more like "If X was running the show, and the forum had no other merits, would X's leadership be enough to make you use it and stick around despite its lack of other merits?" then my answer, at least, will not be informative. I suspect I'm not alone :-).

comment by Dagon · 2017-09-13T14:40:33.569Z · LW(p) · GW(p)

Took most of the survey, but seem to have submitted early somehow.

comment by [deleted] · 2017-09-13T21:06:40.385Z · LW(p) · GW(p)

Re: The SAT question. The SAT's actually reverted back to a score out of 1600. Slight nitpick, given that you mention that the current status quo is out of 2400.

comment by gbear605 · 2017-09-13T17:57:16.245Z · LW(p) · GW(p)

For being diagnosed with depression, do you include only major depressive disorder or do you also include persistent depressive disorder and adjustment disorder?

Replies from: ingres
comment by namespace (ingres) · 2017-09-13T18:52:20.184Z · LW(p) · GW(p)

Any form.

comment by Thomas · 2017-09-13T10:18:02.812Z · LW(p) · GW(p)

Previous session is set to be finished.

Your browser reports that it was used previously to answer this survey. We are resetting the session so that you can start from the beginning.

Click here to start the survey.

I have just pressed Enter after my country's name. Fix this!

Replies from: ingres
comment by namespace (ingres) · 2017-09-13T18:51:59.585Z · LW(p) · GW(p)

I think I'm going to need some more information. Can't fix a bug I can't reproduce.

Replies from: Thomas
comment by Thomas · 2017-09-13T19:53:51.816Z · LW(p) · GW(p)

My country isn't from the list of the default choices. So I type it and pressed Enter. It's all I remember.

Replies from: ingres, ingres
comment by namespace (ingres) · 2017-09-14T06:37:28.639Z · LW(p) · GW(p)

This issue should be fixed now, thanks for your report.

comment by namespace (ingres) · 2017-09-13T20:48:13.663Z · LW(p) · GW(p)

That helps, thank you.

comment by morganism · 2017-09-14T20:32:44.518Z · LW(p) · GW(p)

can't access "font fort" with noScript on. It needs to be expressed in the link html, so i can whitelist it.

Replies from: SaidAchmiz
comment by Said Achmiz (SaidAchmiz) · 2017-09-15T01:46:43.889Z · LW(p) · GW(p)

Hi, could you elaborate on what you mean by "It needs to be expressed in the link html"?

comment by namespace (ingres) · 2017-09-13T07:01:55.650Z · LW(p) · GW(p)

In addition to the email listed at the end of the survey, you can report bugs by replying to this post and I'll keep a log of their resolution:

Wed Sep 13 00:00:55 PDT 2017: Fixed bug where yourmorals.org survey links took user away from survey page instead of opening in new tab. (If this issue resurfaces please let me know.)

I'm also going to go through and fix the same issue in as many other survey links as I can. In the meantime work around by right clicking to open.

Wed Sep 13 21:50:11 PDT 2017: Fixed bug where enter key on text forms would throw out your survey information. Multiple people reported this and it should no longer happen.

Fri Sep 15 13:46:59 PDT 2017: Fixed bug that prevented survey users from commenting on boxes they hadn't checked in question 36.

comment by [deleted] · 2019-06-03T18:13:05.279Z · LW(p) · GW(p)

.

Replies from: Raemon, habryka4, Elo
comment by Raemon · 2019-06-03T23:07:18.827Z · LW(p) · GW(p)

While this is getting discussed – an issue I've been running into is that there's lots of different sub-communities that have surveys, and it's pretty annoying to have to do multiple surveys (for SSC, for LW, for EA, and potential others)

It'd be nice if everyone doing surveys managed to coordinate in such a way that they got to share a lot of baseline demographic questions, and then ask more specific questions relating to their sub-community.

comment by habryka (habryka4) · 2019-06-03T19:44:12.975Z · LW(p) · GW(p)

There haven't been any further, but I would be open to helping run one.

comment by Elo · 2019-06-03T19:30:45.848Z · LW(p) · GW(p)

Nope, hasn't been done.

comment by berekuk · 2017-10-05T13:11:08.264Z · LW(p) · GW(p)

So, what happened?

This post is hidden from Main and the survey "is expired and no longer available", even though the post mentions that it should run for 10 more days. I wanted to share it with Russian LW community, will it be back in some form later?

Replies from: ingres, ingres
comment by namespace (ingres) · 2017-10-07T02:06:34.951Z · LW(p) · GW(p)

Right sorry, I got distracted by life a bit there. I'll write up a post explaining what happened to the LW Survey soon and where I'm planning to go from here.

comment by efenj · 2017-09-13T21:36:19.997Z · LW(p) · GW(p)

Firstly, thank you for the survey and for the option of exporting one's answers!

Questions that I found ambiguous or without a clear, correct answer (for future reference, since changing the survey midway is a terrible idea):

  1. Is it fundamentally important to you that the 'rationality movement' ever produces a measurable increase in general sanity? (i.e, if you were shown conclusive proof it will not you would likely leave)?

What do you answer if you believe that it is fundamentally important, and worth trying, but still unlikely to succeed (i.e. we're probably doomed, but we should still make an effort)?

  1. Do you attend Less Wrong meetups? Yes, once or a few times

Attended once or a few times, in total, or attend once or a few times per year/other reasonable time period?

Replies from: ingres
comment by namespace (ingres) · 2017-09-13T21:43:49.565Z · LW(p) · GW(p)
  1. Yes.

  2. Total, I would think.

Replies from: efenj
comment by efenj · 2017-09-14T02:00:42.407Z · LW(p) · GW(p)

Thanks for the very fast reply!

I interpreted 2 correctly (in line with your reading), for 1, the "you would likely leave" part misled me.

comment by MaryCh · 2017-09-17T14:42:18.163Z · LW(p) · GW(p)

Thank you for compiling another survey!

And on a completely unrelated note, what is it that we actually want to know about ourselves as LW? Surely it can't be the gender ratio. It's not like we don't already know not to post "traditionally feminine" stuff or something. It seems to me that surveys aren't done to achieve some further goal, although the results, of course, are of some curiosity. Sorry if this is counterproductive, I am genuinely interested in the above question.

Replies from: gjm, Elo
comment by gjm · 2017-09-18T12:18:22.193Z · LW(p) · GW(p)

I think it's mostly just curiosity.

There are some things that make useful responses to common criticisms of the community. "You think you're so smart, but you really aren't!" (People reporting "proper" IQ scores and SAT results here consistently come out a couple of standard deviations above average.) "LW is a hive of racist sexist neoreactionary scum and villainy!" (LW consistently comes out more liberal and more feminist than the population as a whole. It has a lot of gay people and a lot of trans people. On the other hand, it doesn't have a lot of women or black people and some specific ideas neoreactionaries like are more popular here than in the world at large.) "You're a bunch of borderline autistics!" (Er, um, yeah, that one seems not to be too far from the truth.)

comment by Elo · 2017-09-17T19:52:15.394Z · LW(p) · GW(p)

Without explaining - the demographic is useful to know.

I think it's good to ask the question - if the question yielded x results what would we do with that information?

For gender specifically non-binary gender is about 10x more common on lesswrong than in the American population. That's worth knowing.

Replies from: MaryCh
comment by MaryCh · 2017-09-18T19:50:38.199Z · LW(p) · GW(p)

But why is it worth knowing?

Replies from: Elo
comment by Elo · 2017-09-20T21:33:05.809Z · LW(p) · GW(p)

For that subset of the demographic there may be use in posts on relevant topics. Just as we have higher (double) depression rates than the normal population, and a post on depression may be relevant to them.