Open Thread - Jan 2022 [Vote Experiment!]

Please tell us what you think! Love it/hate it/think it should be different? Let us know.

I think it's a fine experiment but... right now I'm closest to "hate it," at least if it was used for all posts (I'd be much happier if it was only for question-posts, or only if the author requested it or a moderator thought it would be particularly useful, or something).

• It makes voting take longer (with not much value added).
• It makes reading comments take longer (with not much value added). You learn very little from these votes beyond what you learn from reading the comment.
• It's liable to make the more OCD among us go crazy. Worrying about how other people vote on your writing is bad enough. I, for one, would write worse comments in expectation if I was always thinking about making everyone else believe that my comments were true and well-aimed and clear and truth-seeking &c.

If this system was implemented in general, I would almost always prefer not to interact with it, so I would strongly request a setting to hide all non-karma voting from my view.

Edit in response to Rafael: for me at least the downside isn't anxiety but mental effort to optimize for comment quality rather than votes and mental effort to ignore votes on my own comments. I'm not sure if the distinction matters; regardless, I'd be satisfied with the ability to hide non-karma votes.

I was talking to a friendly recently who is an experienced software developer looking to get into AI safety. Both of us have been reading LessWrong for a long time, but were unclear on various things. For example, where can you go to see a list of all job and funding opportunities? Would jobs be ok with someone with a software engineering background learning AI related things on the job? Would grants be ok with that? What remote opportunities are available? What if there is a specific type of work you are interested in? What does the pay look like?

These are just a few of the things we were unclear on. And I expect that if you interviewed other people in similar boats, there would be different things that they are unclear on, and that this results in lots of people not entering the field of AI safety who otherwise would. So then, perhaps having some sort of comprehensive career guide would be a high level action [LW · GW] that would result in lots more people entering the field.

Or, perhaps there are good resources available, and I am just unaware of them. Anyone have any tips? I found 80,000 hours' career review of AI safety technical research and johnswentworth's [LW · GW] post How To Get Into Independent Research On Alignment/Agency [LW · GW], but neither seems comprehensive enough.

Edit: As an alternative, we could also have some sort of page with a list of people in the field of AI safety who are willing to chat on the phone with those who are looking to enter the field and answer questions. Now that I think about it, I suspect this would be both a) more effective at "converting" new "leads", and b) something that those in the field of AI safety would be more willing to do.

Why do I believe (a)? Having a career guide that is comprehensive enough where you get all of your questions addressed is hard. And there's something about speaking with a real person. Why do I believe (b)? Chatting with people is fun. Especially when you are able to help them. It also is low-commitment and doesn't take very long. On the other hand, writing and (especially) maintaining a guide is a lot of work.

So then, here is a Google Doc: https://docs.google.com/document/d/1XlvUpShIuO_kSFu8IFfhX7lKpr10EypDKXGN-CaS7FM/edit?usp=sharing.

• If you're in the field of AI safety, it would be awesome if you added your contact info.
• If you know someone in the field of AI safety, it would be awesome if you brought this to their attention.
• I just threw this together haphazardly. If someone is willing to take over the project and/or make the doc a little nicer, do something better in Notion, or create a real website for this, that would be awesome. I'd pursue this myself if there was enough interest (I'm a programmer and would build a real website de
• If you are a LessWrong moderator, it'd be cool if you considered linking to this prominently. I feel like that might be a necessary condition for this succeeding. Otherwise it feels like the sort of thing that would rely on word of mouth to know that it exists, and that it probably wouldn't spread well enough to survive long enough.
• If you are someone looking to get into the field of AI safety research, it would be great if you could share your thoughts and experiences, positive or negative, so we can update our beliefs about what the pain points really are.

[deleted]

The idea behind this voting system is to act as a culture-shaping tool: the ballot you see when you hover over the vote buttons is meant to tell you what we think makes for good and bad comments. Ideally, this message comes across even if you aren't voting much.

I've given some thought to the specific axes and reactions, but they should still be treated very much as a first draft. I'm very interested in comparing other people's lists of axes and reactions, and in arguments about what should and shouldn't be included. What makes for a good comment? What should people be paying attention to? What have you wanted to communicate to authors, which you wish you had a button for instead of having to write a whole comment?

A big uncertainty I have about this voting system is how much of a problem the extra complexity is. Is seeing the extra score components on comments distracting? Does having a bunch of extra axes to vote on make voting too time consuming or overwhelming? Feedback on this is appreciated.

And of course, now that we have a setup in place where we can try out alternative voting systems, we're interested in any original ideas people have for different ways of voting on comments that we haven't thought of.

Twitter has announced a new policy of deleting accounts which have had no activity for a few years. I used the Wayback Machine to archive Grognor [LW · GW]'s primary twitter account here. Hal Finney's wife is keeping his account alive. 
I do not know who else may have died, or cryo-suspended, over the years of LW; nor how long the window of action is to preserve the accounts.

Feature idea: Integrating https://excalidraw.com/ into the editor so that users can quickly and easily draw sketches and diagrams. I have been doing so a little bit, eg. this diagram in this post [LW(p) · GW(p)].

I'm a big fan of visual stuff. I think it is pretty useful. And their GitHub repo says it isn't that hard to integrate.

Try out @excalidraw/excalidraw. This package allows you to easily embed Excalidraw as a React component into your apps.

This is a test comment. You may react to it with impunity! Vote at will!

I'd appreciate it if clicking on the regular upvote / downvote didn't open the more complex dialog, and rather just did a simple up / down vote, and instead there was a button to access the more detailed voting. That way, by default, voting is easy and I can ignore the more nuanced system unless I deliberately wanted to use it.

(Also, since we're on the topic of the voting UI, I've mentioned to multiple members of the LW team that strong upvoting is broken on iPad, since the OS says long press = select text. On iPhone, a different gesture is used, but it's activated based on screen size, so it doesn't work on iPad. This should be easily fixable by simply adding a check for OS that makes the double-tap always work on iOS (though things are often not as simple as one may expect). I'm a little frustrated that this hasn't been fixed yet, though I also understand that dev resources are limited)

It took a minute to "click" for me that the green up marks and red down marks corresponded to each other in four opposed pairs, and that the Truth/Aim/Clarity numbers also corresponded to these axes. Possibly this is because I went straight to the thread after quickly skimming the OP, but most threads won't have the OP to explain things anyway. So my impression is it should be less opaque somehow. I do like having votes convey a lot more information than up/down. I wonder if it would be best to hide the new features under some sort of "advanced options" interface.

"Seeks truth" and "seeks conflict" aren't always opposites. For example, it's common for comments to seek harmony instead of either truth or conflict.

If there's going to be a small number of emojis, they should probably be very different colors, like red/yellow/blue/green.

It's easy to write a comment that's net positive overall. It's hard to write one that's separately net positive on each axis. I expect a system like this would lead to me spending more time crafting my comments and posting fewer (better) comments overall.

After this and the previous experiments on jessicata's top level posts, I'd like to propose that these experiments aren't actually addressing the problems with the karma system: the easiest way to get a lot of karma on LessWrong is to post a bunch (instead of working on something alignment related), and the aggregate data is kinda meaningless and adding more axis doesn't fix this. The first point is discussed at length on basically all sites that use upvote/downvotes (here's one random example from reddit I pulled from Evernote), but the second isn't. Given an example post, what does it mean that, say, 15 people upvoted it and 3 people downvoted it?

It means nothing.

There is an assumption that this aggregation actually is useful to the user and I'd like to push back on that. Even ignoring sockpuppeting (hi Eugene) and offsite brigading (hi [REDACTED]), how is a total score of "12" supposed to help me? How does a score of 12 predict whether I'd like this comment or not? Adding a separate agree/disagree sum (like on jessicata's posts) or a set of additional tags (like here) doesn't address this.

Here's a more interesting experiment that's admittedly much more disruptive and difficult to pull off: leave the upvote/downvote buttons, but completely hide total karma scores entirely from the user. Then do something like surface the comments in the order that LessWrong predicts the viewing user will upvote/take no action/downvote. My downvote might uprank a comment for someone else, making voting more valuable for everyone. This still feels vaguely Goodheart-y and is more of a starting point, but seems much more valuable than the current system.

I like the idea of using the Open Thread for testing new karma systems.

Adding multidimensionalilty to it certainly seems like a good idea. In my experience, karma scores on comments seem to be correlated not just to quality of content but also to how well it aligns with the community narrative, to entertainment value, to the prior status of the commenter, and even to the timing of the comment relative to that of the post. Disentangling these would be helpful.

But then, what is it we really want karma to represent? If community members are not vigilant in how we rate things, the karma system is ripe for Goodharting. It's easy to feel compelled to try whatever it takes to get that karma counter to increment.

In my opinion, karma ought to represent how much a comment or post should be seen by other members of the community, in terms of how useful it is for promoting good ideas specifically or rational thinking/behavior generally. Upvotes/downvotes are only (somewhat weak) Bayesian evidence for or against this.

Some tiny bugs on this Walled Garden [? · GW] LW page:

• Clicking on the "Walled Garden" banner leads to this URL [? · GW], which doesn't exist.
• The second sentence here is incomprehensible: "If you have an event invite link, please use that to enter. If you have been granted full-access, to log in."

Some thoughts on the individual axes:

• True/False -- this seems not always applicable and similar to agree/disagree in the cases where it is? Could be missing something, but I don't think I like this.
• Hits the Mark/Misses the point -- SlateStarCodex had the "two of {true/kind/necessary}" policy; going off topic was fine as long as it was kind and true. My impression has been that something like this is true on LessWrong as well, which seems good? I don't think this one is a good idea at all, at least not if people vote on it all the time.
• Clear/Muddled -- better; seems like a relevant axis and a universal good. That said, it's much harder to be clear on some topics than others, so I feel like people ought to vote on relative clarify and I don't know if they will.
• Seeks Truth/Seeks Conflict -- better for the same reasons as above and doesn't have the same problem
• Emojis -- honestly no idea yet

Overall, I feel like 5 axes & emojis are obviously going to be too much. I also echo Zach Stein-Perlman wrt the negative effect on anxiety.

I quite like the 2-axis system of overall & agreement. I guess what I would like to see more is something like that system plus maybe parts of [this minus the first two axes] if there is a way to make it far less complex. Spitballing an idea here, perhaps you could tweak the interface such that voting on the other axes is not the norm. Maybe if you first have to click a button to do it and then it's only shown above vote strength 5 or something, so it needs two people or you need to strong-vote, and the people who disagree that the post is unusual along that axis can vote into the opposite direction to make it disappear again. That way, if you really think a post is too conflict-seeking, you can express that anonymously, but it doesn't clutter up every post.

I also think it should preferably always be possible to avoid negative scores if you try, for anxiety reasons, which is perhaps an argument against the clear/muddled axis but not the conflict/truth one.

Either way, I enthusiastically support the experiment!

Epistemic Status: groping around at an event idea I hope others are interested in

I don't know how to communicate this yet, but there's a ritual I want to do with friends this summer. The following describes some inspirations and gestures toward the general aesthestic.

• It was part of my step-family's lore to learn camping skills and wilderness survival, at one point even giving little "merit badges" for demonstrating mastery. With a similar spirit they would also host summer 'art shows' where'd we'd learn about a different culture and put things we made that year (or day) on display for the family to see.
• Judaism has this week-long celebration that involves building a simple outdoor shelter which one eats in, sometimes sleeps. The shelter itself symbolizes the fragility and transience of life and one's dependence on God.
• Burning Man is a temporary Art-themed city that springs up for a week in the middle of the nowhere and gets torn down cleanly at the end. The standard middle of nowhere happens to be in a freaking desert, so it's got this strong ethos of pushing one's limits individually but also looking out for each other. It emphasizes raw experiences and co-creation, eschews symbols of consumerism. Whole lot of drugs, music, sex, and weirdness.
• Midsummer bonfires or maypoles. For drinking, dancing, singing around. Associated with "fertility", aka event of the year for flirting and/or hookups.
• How To Make Everything youtube series: "Everything we use comes from 8,000 generations of collective innovation and discovery. But could an average person figure it all out themselves and work their way from the stone age to today?".
• snippets from the sequences
  • Making History Available [LW · GW] - "I should immerse myself in history, imagine living through eras I only saw as ink on paper. ... I had to overcome the false amnesia of being born at a particular time. I had to recall—make available— all the memories, not just the memories which, by mere coincidence, belonged to myself and my own era. ... So many mistakes, made over and over and over again, because I did not remember making them, in every era I never lived . . ." 
  • What is Evidence [LW · GW], Entangled Truth, Contagious Lies [LW · GW] - rationality as entangling oneself with truths, and truths being heavily entangled with each other.
  • Truly Part of You [LW · GW] - "How much of your knowledge could you regenerate? From how deep a deletion? ... A shepherd builds a counting system that works by throwing a pebble into a bucket whenever a sheep leaves the fold, and taking a pebble out whenever a sheep returns. If you, the apprentice, do not understand this system—if it is magic that works for no apparent reason—then you will not know what to do if you accidentally drop an extra pebble into the bucket. That which you cannot make yourself, you cannot remake when the situation calls for it."
• Doing stuff badly and having fun with it, like a child. Your work is going to suck. But it's yours in a way many hyper-optimized alternatives are not.
• Hackerspace/Makerspace: Community-operated work spaces where people with common interests (often in technology, science, and digital art) can meet, socialize, shbrare infrastructure, and collaborate.

Am stalled on figuring out what the minimum viable parts are

I don’t like this voting feature on mobile. It makes it impossible to press the normal vote arrow without zooming in because I keep fat-fingering something other than the regular vote arrow.

I'm not a fan of this voting system tbh. I guess I just find it too distracting.

Reaction-ballot voting has a "you make what you measure" feel to me.

7. You make what you measure.

I learned this one from Joe Kraus. [3] Merely measuring something has an uncanny tendency to improve it. If you want to make your user numbers go up, put a big piece of paper on your wall and every day plot the number of users. You'll be delighted when it goes up and disappointed when it goes down. Pretty soon you'll start noticing what makes the number go up, and you'll start to do more of that. Corollary: be careful what you measure.

— http://www.paulgraham.com/13sentences.html

If people can vote on your comments along an axis of eg. seeking truth vs conflict, I expect that users will spend more effort to seek truth rather than conflict.

However, there is a risk of unintended consequences. For example, the presence of the truth vs conflict axis might push people away from babble-y and contrarian comments. I actually expect that this would happen in a non-trivial way with the current axes. But if there was an additional axis like "bold vs timid", I think that would offset the effect. Eg. in the sense of how sticking your neck out is a rationalist virtue, as opposed to using language like "X might be the case".

I love it. 

I reviewed my voting proposal [LW(p) · GW(p)] and see my suggestions covered except for the checkmark suggestion.

[deleted]

I kind of feel like there should be a funny/not funny axis. Sometimes I read a good joke or a fun take in a comment, and I would like to signal I liked it, but the overall karma does not seems like a good way to signal that.

Also true and hits the mark do not seem orthogonal to me. Can something be false and still get the point ?
 

1. Experiments

The missed opportunity to be able to vote on the post itself this new way stands out - so I'll put it here: 🎉!

2.

The old voting problem is still present: Roughly, the longer the comment (or post), the more likely the same person has different views of parts.

3.

The (modular) anonymity aspect (and the dichotomy aspect) limit some combinations as being clearly from the same person.

Seeks Truth/Seeks Conflict

Combine both and you can get something. Seeks Cruxes? (Currently blocked by exclusive restriction.)

Skeptical + Enthusiastic*

Not quite the same as 'important if true'. Maybe: 'if this works out it could help progress (think cruxes)', or 'I'm not sure that'll work, but if it does it'll be amazing.'

Also could be Experiment/experiment related. (In that there's a sense it could go both ways.)

(Currently blocked by exclusive restriction.)

*I read these as being intended as opposites, though they weren't set to be exclusive. Empathy/Surprise, less so, but still a little bit.

Hits the mark + Misses the point:

You hit the dead center...of a different target.

I could see these being tricky, in that, people arguing about what the point is, happens. Also sometimes a really unusual definition is employed, but not given, and that throws off everyone discussing the implications. (Not that this happens a lot, but I saw it happen recently. Another case of 'important stuff in the comments, that the post should be edited to reflect, if only via a 'Edit: by _ I meant _.' added to the end of the post.)

Another weird bug, found here [LW(p) · GW(p)]. Also see the comment by Zack_M_Davis.

The bug: this is an intentionally broken unclickable link. It's supposed to link to "http:// asdf", and it seems like leaving a space in it is enough to make it unclickable.

Tiny bug with the LW 2020 Review [? · GW] progress bar: The dates on the progress bar disappear depending on horizontal window size. At its full size, the bar ends on "Feb 1st"; at a slightly smaller window size, the bar ends on "Final Voting", with "Feb 1st" out of view; and once the window is small enough, and the progress bar is displayed at the top, then it no longer displays dates at all.

Here is a gif of the problem.

epistemic status: felt sense poetry

Think about a tree. A tree with roots going deep into the ground, and leaves spread out to catch as much sun as it can. Hold that tree in mind.

We often dream of leaving the earth and solar system under our own power. It's an important goal. It's not, however, immediately achievable. We are, for now, tied to this pale blue dot. Sol III, Terra, the world that birthed us. And when we do leave, we will take much of it with us. Some of it we will take intentionally, because we're sentimental like that. But some we will take in the marks of growing up on this pale blue dog, in *what kind of beings our first world made us*.

I want to dedicate a time of the year to think about the environment that we live in, how it shapes us and how we shape it. We ought to know the soil we grow in, and know it well. A general purpose rationality must produce local knowledge, specialized to the time and space and life we are living. We must dig our roots deep, and spread our leaves wide.

We talk a lot about the general theory of rational agents. It informs how you learn true things and make good choices. But, you are not the general example of an agent. Your mind did not spring up from first principals. Evolutionary history, cultural memetics, and human institutions also play a role in the way you learn true things and make good choices. That which is at your roots effected what the truth to be found and good choices to be made even are, thousands of years later.

The tree is a metaphor. Grand forces of optimization have existed before you, and will likely exist after you. Seek them out. Use what they can give you. Dig your roots deep.

And spread your leaves wide. Plants are often portrayed as passive; that's incorrect. They move. They move a great deal, efficiently and slowly, in order to catch the sun. In the spirit of a plant, you need not chase after every opportunity. You can position yourself well, and take the chances as they come to you. Have patience, not passivity. And spread your leaves wide. Catch every ray of sunlight and harness it. Drink in the present moment, and turn it into something awesome.

• What's something you know about the environments that you live in, or that shaped you? Your city. Your ecosystem. Your history. Your economy. Your community. What don't you know about them that you would like to?
• What serendipitous chance might you hope to catch, this month? What would you be prepared to act on if it happened?

Bug: In this [? · GW] post, there's one footnote, but its return-from-footnote link does not work. That is, when I click on it, the browser screen doesn't move back to the footnote link. However, when I load that return-from-footnote link [? · GW] in a new tab, it does correctly center the screen on the footnote link.

My immediate (kneejerk System-1-ish emotional) reaction to the experimental voting popup is along the lines of "meh, too much effort, won't bother".

My slightly less immediate reaction turns out to be much the same. So e.g. I think "I should give this a try", take a look at the comment currently presented to me first, and ... well, there are now 9x as many things to decide about it as before (overall opinion, 4 "axes", and 4 possible reaction emoji), and all but the first feel as if they require substantially more mental work to arrive at a useful opinion about, and I just don't wanna.

But!

In practice, the way I usually use the LW karma system (other than "implicit" uses like reading the comments in the order the UI presents them to me in, which is affected by karma) is something like this: I vote only when something I read strikes me as especially good or bad, in which case I consider briefly whether it's good/bad enough to be worth voting on, and whether if so it warrants a strong up/down-vote. This process seems like it might cope fine with the more complex interface -- if I've been particularly impressed or unimpressed by something I'll have some sense of what about it is impressive or unimpressive to me, and having a finer-grained way to express that seems like a gain rather than a loss.

So it may be that the answers to the questions "is explicitly deciding what to do with a particular post or comment easier or harder with this experiment?" and "is voting as I actually do it easier or harder with this experiment?" are substantially different.

[EDITED to fix an inconsequential typo.]

I'm not sure what it says about LW that in the current Open Thread there is only one comment that isn't either (1) about the voting-system experiment or (2) deleted.

(And I slightly suspect that that one comment may have been inspired at least slightly by the voting-system experiment, not that there's anything at all wrong with that.)

I actually really enjoyed these voting axes.

I wouldn't be opposed to them being rewritten, but I really liked being able to separate these things out. I will say that not knowing whether or not I voted on an axis from overview is annoying (like how you can see green or red arrows on a post when you regular-vote it.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8297542/ (bias: authors include employees of one of the companies being evaluated, Xpert) might help us choose between all the different tests floating around for sale. It was published on July, 2021 and discusses products from the following firms:

• Accula
• BioFire
• cobas
• Cue
• ID NOW
• Lucira
• Xpert
• Visby

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7785428/ (Bias: "Cue Health provided readers and cartridges for the study.") was published on May 2021 and evaluates Cue Health.

I agree that voting might be little bit annoying. 

On the other side, it could potentially make the search for specific qualities of comment much easier if automated (by sorting). (E.g. "Now I am not in the mood for solving difficult concepts so I want something with high clarity evaluation." or "Now I am too tired to argue/fight so I want something empathic now.")

Is there a post as part of the sequences that's roughly about how your personality is made up of different aspects, and some of them you consider to be essentially part of who you are, and others (say, for example, maybe the mechanisms responsible for akrasia) you wouldn't mind dropping without considering that an important difference to who you are?

For years I was thinking Truly Part Of You [LW · GW]was about that, but it turns out, it's about something completely different.

Now I'm wondering if I had just imagined that post existing or just mentally linked the wrong title to it.

The UI for the reactions works pretty well on iPhone, the only issue is that it's tricky to dismiss the dialog, though it can generally be done with less than 10 seconds of fiddling (usually closer to 1 or 2 seconds). If there was a button to dismiss the dialog, that could make it a lot smoother to use (and should work well on other platforms as well, even if it's not strictly needed on other platforms)