2022 Less Wrong Census/Survey: Request for Comments

The "Profession" question's text presupposes that respondents work in some "academic field". I think deleting the word "academic" would be an improvement, since it would e.g. avoid the risk that someone who works as a farmer thinks "oh, this question is not for the likes of me" rather than just putting in "Farming" or "Agriculture" in the write-in slot at the end (which I assume would be more helpful).

The "Profession" question says "If more than one, please choose most important" ... but then, unlike most other questions, has non-mutually-exclusive checkboxes rather than mutually-exclusive radio buttons. I think it should either enforce "choose only one" or else have text that embraces the option of choosing more than one.

There are a few double-spaces, which are completely harmless but look kinda ugly. I see three in the "human biodiversity" question and one in the "Great Stagnation" question. (Those are the only questions where I noticed them.)

Perhaps the "SRS" question should have finer-grained answers. "Every day", "Frequently", "Occasionally", "Never" or something. Or perhaps the question should have a word like "regularly" or "at least once a week" in it. (Every couple of years, I remember the existence of Anki and make a desultory effort to learn some stuff using it, but the habit never really sticks. You probably want a "no" answer from me, but strictly speaking my answer to the question as written is "yes".)

There is a thing saying "Section 9: More complicated probability questions", after which there is only one probability question, and actually even that one is not exactly a probability question; after that are a bunch of not-probability questions. I suspect that it should be something more like "Section 9: Other questions", and if so then probably either the words that follow should be tweaked or else they should be moved to the previous section heading.

I don't know how common it is for answers to questions in the "Probabilities" section to be misleading because answerers give probabilities rather than percentages despite the nice clear instructions. If it's not uncommon, I wonder whether it would be an improvement to make the instructions say explicitly that one should include the percentage sign. Presumably someone who does so can reasonably confidently be assumed to be giving a percentage rather than a probability, and someone who doesn't is at least known not to have read the instructions carefully, so if nothing else those responses could be discarded.

BUT maybe that would throw off whatever automated analysis Google Forms does? My hazy recollection is that whatever automated analysis Google Forms does is pretty terrible anyway, but I'm not very sure about that. (... I just checked the ACX2022 survey results and yes, it's terrible; it just lists all the responses and doesn't turn them into summary statistics or histograms or anything. So including percent signs doesn't seem like it would do any harm to speak of.)

It would be nice if the "Time in Community" question had a note saying something like "If you have been here since Less Wrong first got started, put X months" rather than making us look up when that was.

(According to Wikipedia, which as we all know is never wrong about Less Wrong, the first incarnation of LW-as-such was in 2009-02, which would mean it was just under 14 years = 168 months ago; Overcoming Bias started in 2006-11, so if you want to count the OB days too the longest possible interval would be 16y2m = 194 months.)

Some years ago, I think the "IQ" question (or maybe one of the various questions aiming to quantify brainpower) didn't discourage less-rigorous answers. Might there be value in asking both for properly measured IQ values and for estimates, results of crappy internet tests, etc.? It might maaaybe be possible to get some idea of the relationship between proper measurements and estimates by seeing what properly-measured IQ range any given estimated IQ range "looks like" in terms of answers to other questions.

But arguably the LW community is too focused on its own intelligence, and on IQ in particular as a measure thereof, and it would be better to have less of this rather than more :-).

The "Religious Denomination" question is arguably mistitled, since e.g. "Hindu" and "Traditional Chinese" are not denominations as usually understood.

More substantially, I wonder whether it would be a win to modify the question so that it's meaningful for some non-religious people: something like "If you are not now religious but were in the past, please answer as you would have done when most recently you were religious." It would still need an option for "not religious" in case someone who was never religious accidentally clicks on one of the radio buttons. Note that this would make the personal-religion questions more parallel to the family-religion questions. But I'm not sure whether this would actually provide sufficient extra value to be worth the extra complexity. 

On the ACX 2022 survey, the "Politics" question deliberately adopted different wording for the "Neoreactionary" option from the one it's used in the past and that's also used here. Here's Scott, talking about the results of that survey:

The biggest effect is that many fewer people identify as neoreactionary, but I'm pretty sure that's because I changed the question wording. It was previously "neoreactionary, such as Singapore", but I thought that in real life nobody thinks of Singapore as NRx, and this was probably attracting a lot of Singapore fans who had no idea what it meant otherwise, so I changed it to "neoreactionary, such as the writings of Curtis Yarvin" and it dropped by more than half. Unfortunately this means I can't track actual variations in neoreaction popularity over time, which I'd be interested in knowing. There was no similar change to the definition of alt-right, but it dropped by about 33%.

I think that (1) he is correct that "neoreactionary" doesn't really mean "like Singapore" but also (2) the change makes inter-year comparisons difficult. I wonder whether the Right Thing would be for the LW survey to keep the old wording this year and then change next year to Scott's new wording. (Then if you want to compare LW 2020 with LW 2024, you can try to relate LW 2020-2022 to LW 2023-2024 despite the wording change by guessing that LW's 2022/2023 change in support for neoreaction here is comparable to ACX's change in the same pair of years, which you can see because ACX uses the same wording for those years.)

The "Living With" question has among its answer-options "With family" and "With partner/spouse". This could be trying to draw two different distinctions: (1) with partner/spouse only versus e.g. with children, or (2) with partner/spouse and possibly also children versus with parents. I think 2 is more likely, and expect most others to interpret it that way, but I'm not sure it's so much more likely that no one will interpret it differently.

I wonder whether this might actually be better with a few not-mutually-exclusive tickyboxes: "Do you live with any other people? Check all that apply", and then "Parent(s)", "Partner/spouse", "Children", "Others". Or something like that.

Alternatively, and assuming I'm right about which meaning was intended: "Alone" / "With parents and/or siblings" / "With partner/spouse and/or children" / "With roommates" (though that last one feels a bit too specific to me).

what's meant by "life"? If the question is "something recognized by a majority of humans as life", I'd give a very different answer than if the question is "something recognized by any human as life".

There's also some e-prime-flavored breakage to "is" in the probability questions, perhaps intrinsic to all probability questions. For instance, there are some people who go through life with absolute certainty that the government is controlled by camouflaged reptiles, and other people who find reptiles the best available metaphor for predicting how the government will behave. I believe it's overwhelmingly likely that tissue samples and full-body scans of all politicians would show them to be biologically indistinguishable from other humans, yet it is simultaneously true that some people live in a world where the politicians are "really" reptilian and the tissue samples were faked somehow. What probability in that box would accurately communicate this view?