Dealing with trolling and the signal to noise ratio

Seconded. The first time that I saw any indications of LW having a troll problem was a couple of days back, when people started complaining about us having one.

I'm not even sure he's a troll--my impression is that he isn't very fluent in English and inadvertently got himself anti-LW-mindkilled.

This rule is asinine.

Indeed. But why is our Rational Leader overreacting to what would seem a minor issue? The question bears analysis. Have some of the public exposures of SIAI left him feeling particularly vulnerable to criticism?

This seems to assume that having correct tags will reduce noise, actual or perceived. That's not clear to me. I never look at the tags when deciding what to read, only the titles. Why would accurate tags be useful?

One thing we can do now is use the wiki to index articles.

You've made me understand the root of one of my own dissatisfactions with the current system. If I look through my post history and roughly group my posts into bins based on how I would summarize them, this is what I see:

• Silly posts in the HP:MOR threads: ~ +20 karma

• Posts of mine having little content except to express agreement with other high-karma posts: ~ +10 karma

• Important information or technical corrections in serious discussions: ~ +1 karma

• Posts which I try to say something technical which I retrospectively realize were poorly worded but could have been clarified if someone pointed out an issue instead of just downvoting: ~ -5 karma

Perhaps I exaggerate slightly but my point is that if I were to formulate a posting strategy aimed at obtaining karma, then I would avoid saying anything technical or discussing anything serious and stick to applause lights and fluff.

On top of this, I tend to watch how the karma of my most recent comments behaves, and so I notice that, for example, a comment might have +5 upvotes and -3 downvotes, with no replies. This is just baffling to me. Was there something wrong with the post that three people noticed? Were the three separate things wrong with it? Was it just a response to the tone? What about the upvotes, is it being upvoted because of the witticism at the end, or because of the technical content in the middle? My point is something like Slashdot has a system where things are voted "funny" or "insightful" would be infinitely more useful.

No web discussion forum I know of has filtering capabilities even in the ball park of Usenet, which was available in the 80s. Pitiful.

I strongly share your opinion on this. LW is actually one of the better fora I've come across in terms of filtering, and it still is fairly primitive. (Due to the steady improvement of this forum based on some of the suggestions that I've seen here, I don't want to be too harsh.)

It might be a good idea to increase comment-ranking values for people who turn on anti-kibbitzing. (I'm sure other people have suggested this, so I claim no points for originality.) ...What a great feature!

(Of course, then that option of "stronger karma for enabled anti-kibbitzers" would give an advantage the malevolent people who want to "game the system" who could turn it on and off, or turn it on on another device, see the information necessary to "send out their political soldiers" and use that to win arguments at a higher-ranking karma. Of course, one might want to reward malevolent players, because they are frequent users of the site, who thus increase the overall activity level, even if they do so dishonestly. They then become "invested players," for when the site is optimized further. Also, robust sites should be able to filter even malevolent players, emphasizing constructive information flow. So, even though I'm a "classical liberal" or "small-L libertarian," this site could theoretically be made stronger if there were a lot of paid government goons on it, purposefully trying to prevent benevolent or "friendly" AGI that might interfere with their plans for continuing domination.)

A good way to defeat this would be to "mine" for "anti-kibbitzing" karma. Another good idea would be to allow users to "turn off karma." Another option would be to allow those with lots of karma to turn off their own karma, and show a ratio of "possible karma" next to "visible karma," as an ongoing vote for what system makes the most sense, from those in a position of power to benefit from the system. This still wouldn't tell you if it was a good system, but everyone exercizing the option would indicate that the karma-based system was was a bad one.

Also, I think that in a perfect world, Karma in its entirety should be eliminated here. "One man's signal is another man's noise," indeed! If a genius level basement innovator shows up tomorrow and begins commenting here, I'd like him to stick around. (Possibly because I might be one myself, and have noticed that some of the people who most closely agree with certain arguments of mine are here briefly as "very low karma" partipants, agree with one or two points I make, and then leave. Also, if I try to encourage them but others vote them down, I'm encouraged to eliminate dissent, in the interest of eliminating "noise." Why not just allow users to automatically minimize anyone who comments on a heavily-downranked already minimized comment? Problem solved.)

LessWrong is at risk of becoming another "unlikeliest cult," to the same extent that Ayn Rand Institute became an "unlikely cult." (ARI did, to some extent, become a cult, and that made it less successful at its intended goal, which was similar to the stated goal of LessWrong. It became too important what Ayn Rand personally thought about an idea, and too unimportant what hierarchical importance there inherently was to the individual ideas themselves. Issues became "settled" once she had an opinion on them. Much the way that "mind-killing" is now used to "shut down" political debate, or debate over the importance of political engagement, and thus cybernetics, itself.)

There are certain subjects that "most humans in general" have an incredibly difficult time discussing, and unthinking agreement with respected members of the community is precisely what makes it "safe" to disregard novel "true" or "valuable" solutions or problem-solving ideas, ...rare as they may admittedly be.

Worse still, any human network is more likely to benefit from solutions outside of its own area of expertise. After all, the experts congregate in the same place, and familiarize themselves with the same incremental pathways toward the solution of their problems. In any complex modern discipline this requires immense knowledge and discipline. But what if there is a more direct but unanticipated solution that can arise from outside of that community? This is frequently the case, as indicated in Kurzweil's quote of Weiner's "Cybernetics" in "How to Create a Mind."

It may be that the rise of a simple algorithm designed by a nanotech pioneer rapidly builds a better brain than AGI innovators can build, and that this brain "slashes the gordian knot," by out-thinking humans and building better and better brains that ultimately are highly-logical, highly-rational, and highly-benevolent AGI. This constitutes some of the failure of biologists and computer scientists to understand the depth of each others' points in a recent Singularity Summit meeting. http://www.youtube.com/watch?v=kQ2snfsnroM -Dennis Bray on the Complexity of Biological Systems (Author of "Wetware" describing computational processes within cells).

Also, if someone can be a "troll" and bother other people with his comments, he's doing you a small favor, because he's showing that there are weaknesses in your commenting system that actually rise to the level of interfering with your mission. If we were all being paid minimum wage to be here, that might represent significant losses. (And shouldn't we put a price on how valuable this time is to us?) The provision of garbled blather as a steady background of "chatter" can be analyzed by itself, and I believe it exists on a fluid scale from "totally useless" to "possibly useful" to "interesting." Also, it indicates a partial value: the willingness to engage. Why would someone be willing to engage a website about learning an interesting subject, but not actually learn it? They might be unintelligent, which then gives you useful information about what people are willing to learn, and what kinds of minds are drawn to the page without the intelligence necessary to comprehend it, but with the willingness to try to interact with it to gain some sort of value. (Often these are annoying religious types who wish to convert you to their religion, who are unfamiliar with the reasons for unbelief. However, occasionally there's someone who has logic and reason on their side, even though they are "unschooled." I'm with Dawkins on this one: A good or bad meme can ride an unrelated "carrier meme" or "vehicle.")

Site "chatter" might normally not be too interesting, and I admit it's sub-optimal next to people who take the site seriously, but it's also a little bit useful, and a little bit interesting, if you're trying to build a network that applies rationality.

For example, there are, no doubt, people who have visited this website who are marketing majors, or who were simply curious about the current state of AGI due to a question about when will a "Terminator" or "skynet"-like scenario be possible, (if not likely). Some of them might have been willing participants in the more mindless busywork of the site, if there had been an avenue for them to pursue in that direction. There are very few such avenues on this "no nonsense" (but also no benevolent mutations) version of the site.

There also doesn't appear to be much of an avenue for people who hold significant differences of opinion that contradict or question the consensus. Such ideas will be downvoted, and likely out of destructive conformity. As such, I agree that it's best to allow users to eliminate or "minimize" their own conceptions of "what constitutes noise" and "what constitutes bias."

I'm much more comfortable with this sort of intervention as a "We think this will improve the forums, let's test this for a month or two" rather than "Lesswrong sucks but this will fix it guys, trust us"

It's not a huge burden, but we're already seeing some negative effects worth discussing. For example I have now twice paid the 5 karma penalty replying to downvoted comments which were not trolling at all; they were downvoted because people disagreed with what they were proposing.

If we decide to wait two weeks, we need to decide on specific criteria that we will judge in two weeks' time to decide whether to modify or remove the new feature. If the new feature stays anyway because a few people decide unilaterally, then we might as well discuss it now.

If we're seriously having troll issues, then wiki + trolls = edit wars. Also polarization over schools of editing styles, the way Wikipedia has its Deletionists.

If most of the topics will generate disagreement, growing a wiki make this site less dynamic. Or maybe is fine to change(or improve) definitions all the time.

"Technical solutions for social problems almost never work."

As far as I understand, the "noobstories" list is all the articles submitted by new accounts in reverse chronological order, while HN Classic is the current front page sorted according to the votes of "old" members.

The most obvious example of trolls right now is this post and some of its comments, although as far as trolling goes, neither are very effective.

I might go and figure out how to get some data off of LessWrong commenting system, to try and determine a good indicator for troll-nature.

Master, you have mediated on this for under five minutes, so I wish to ask two things:

• Does not asking about what has the troll-nature bring one closer to the troll-nature?
• If you meet a Socrates on the road does it have the troll-nature?

The proposals here exist outside the space of people who will "solve" any problems that they decide are problems. Therefore, they can still follow that advice, and this is simply a discussion area discussing potential problems and their potential solutions. All of which can be ignored.

My earlier comment to the effect of "I'm more happy with LessWrong's forum than I am unhappy with it, but that it still falls far short of an ideally-interactive space" should be construed as "doing nothing to improve the forum" is definitely a valid option. "If it aint broke, don't fix it."

I don't view it as either totally broken, or totally optimal. Others have indicated similar sentiments. Likely, improvements will be made when programmers have spare time, and we have no idea when that will be.

Now, if I was aggressively agitating for a solution to something that hadn't been clearly identified as a problem, that might be a little obnoxious. I hope I didn't come off that way.

Nesov, you are a particularly active and helpful moderator. I'm less familiar with how much effort is invested by other moderators. I believe you could do this well, but I'm not sure this solution can be scaled, or even run without you (right now).

moderators could be given the power

By whom?

Changes to a functioning system that is in use should be done with care akin to pruning a bonsai tree, not by introducing sweeping changes that are then potentially scaled back.

That would only make sense for posters with, say, negative karma in the last month. Otherwise this results in (self-)censoring of controversial comments.

I very much agree with your proposal, it should have been an obvious first step (and if there had been some public discourse in which you would probably have suggested it, it might well have been). Upon unsatisfactory results, it could have been escalated to a more profound change.

How long is it going to take for some forum regulars to pick up a dedicated downvoter with two sockpuppets, strongly impinging the discussion on their new +0 -> -3 comments, at least temporarily? The karma hit is a negligible inconvenience for old-timers, but the temporary obstruction of their conversational flow isn't. Empowering trolls inadvertently, how ironic. Who would have thought there are such unforeseen consequences when non-professionals implement such changes without discussions? Why, I suspect you would have, as someone experienced with software design failure modes.

It is a bit disconcerting that such an a-priori type proposal was implemented not only without considering expertise from LW's very own base of professionals, but without first gathering some evidence on its repercussions as well. From the resident Bayesianists, you'd think that there'd be some more updating on evidence through trials first, e.g. by implementing similar (such as yours) but less impactful changes first.

Concerning your software development paradigm:

With the caveat of not having spent a long time in the software industry, there is an argument for the converse case as well.

While I'm all for using k-CNFs, DNFs and all sorts of normal forms, penalising lines of code can easily lead to a shorter, but much more opaque piece of software, regardless of doxygen or other documentation. Getting rid of unnecessary conjuncts in a boolean formula sounds good in theory, but just spelling out each case, with e.g. throwing an exception and commenting that this or that case should not happen, can make it much easier for someone else to follow along your logic.

A maximally compressed pile of ultra efficient goodness, resembling a quine in terms of comprehensibility, would satisfy your "minimize the code" requirement, yet make the code less accessible, not more so.

But I'm happy to consider your testimony to the contrary. As the saying goes:

All theory, dear friend, is gray, but the golden tree of life springs ever green.

My estimate of the general intelligence of the subset of LWers who replied to this post has gone way down.

It seems like it's your estimate of the programming knowledge of the commenters that should go down. Most of the proposed solutions have in common that they sound really simple to implement, but would in fact be complicated - which someone with high general intelligence and rationality, but limited domain-specific knowledge, might not know.

Should people who can't program refrain from suggesting programming fixes? Maybe. But maybe it's worth the time to reply to some of the highly-rated suggestions and explain why they're much harder than they look.

(I agree with your proposed solution to attempt simplifications.)

There seem to be two functions of this discussion:

• come up with practical solutions
• diagnose the problem

Terrible to implement comment tweaks can still spur helpful discussion. A poster may have articulated a problem or an incentive in a way most of us haven't considered yet. Not everyone who has an interesting description of the problem may have that much coding-fu. Better to throw up your diagnosis without worrying too much about the cure and let everyone critique, counter-suggest, and fix the implementation.

My estimate of the general intelligence of the subset of LWers who replied to this post has gone way down.

Why was it so high to begin with? What mistakes do you think you made? (Honest questions.)

Downvoted for putting more than one suggestion in a single comment.

Punish me for this anti-social act if you must, but as one of the dudes who tries to act after reading these suggestions (and tries hard to discount his own opinion and be guided by the community) this practice makes it much harder for me to judge community support for ideas. Does your comment having a score of 10 suggest 2.5 points per suggestion? ~10 points per suggestion? 15 points each for 3 of your suggestions and --35 for one of them (and which one is the -35?)?

Can we please adopt a community norm of atomicity in suggestions?

I think #1 is the way to go here, and the only method that will have any effect in most cases.

Since the main problem with trolling is drowning out of recent comments

I agree that if I try to extract coherent beliefs from Eliezer's claim, particularly the claim that people are fleeing the invisible threads, this is what I must conclude.

But I'm not sure I should try to extract coherent beliefs from Eliezer's claims. Do you directly claim this? Do you agree with his claim that trolling has increased in recent months? Do you think invisible comments are a good proxy for trolls?

I stopped looking at recent comments long ago for reasons of volume. I think it keeps large threads going. But I think large threads tend to be equally worthless, regardless of average karma or karma of the initial comment.

No. The toll applies to commenting on posts that are below -3, not having comments on posts that are below -3.

This would be an improvement since then one's karma would still remain in principle obtainable by summing the karma of all one's comments and posts. But then, why have the arbitrary numbers -3 and -5? Wouldn't it be better if a reply to a negatively rated comment started at the same karma as the parent comment? Smooth rewarding schemes usually work better than those with thresholds and steps.

(I still don't support karma penalties for replies in general.)

In some situations it may be worth replying to a comment with negative value.

Imagine a comment which was made in a good faith, but just happens to be incredibly stupid, or is heavily downvoted for some other reason.

Now imagine a reply that contains just a word or two and a hyperlink to an article which explains why the parent comment was wrong. Does this reply deserve an automatic downvote?

Generally: It is a bad idea to think about one specific example and use it to create a rule for all examples. For instance, not all negative-karma comments are trolling; yet we create a rule for all negative-karma comments based on our emotional reaction to trolling.

There should be a different discussion forum which is readable to all but can only be posted in by those with over, say 1000 karma.

Actually I'd find restrictions on who can or can't on vote on the comments to be a more interesting option. What would a forum look like if only those with over 1000 karma on LW could vote?

On prior forums I have been on, attempts to split into a only some posters and all posters forums have ended badly.

When there are enouph high class posters, everything goes into the high class forum and the open forum collapses leaving no worthwhile "in" for new users. When there are too few high class users, everyone double posts to both forums in order to get discussion and you wind up with a functional 1 forum system except with lots of links and more burden and top level menus.

I have not seen an open / closed forum system with exactly the goldilocks number of high class users to maintain stable equilibria in both forums.

The main objection I'd have to that is that standards might drop in the "open" section to the point where some users might just start ignoring it, fracturing the community.

Alternately: a (very low) karma requirement for comments and posts to the discussion subreddit (~20?), a somewhat higher karma requirement for comments and posts to Main (~200?), and a newbie subreddit for new posters to post in. Regular newbie forum topics would be welcome threads and stupid question threads; anybody could post in the newbie forum, so there would responses from LW veterans, and anybody would still be able to ask Stupid Questions. (Meetup threads possibly should go there too.) (Depending on how many other topics there are, it might be worthwhile for every introduction, meetup, and question to get its own thread there.)

I like the proportionality idea but still want to be able to express a vote that says "I want less posts like this" without also saying "I don't want replies to this". My proposal is that replies to comments at -5 or better are free, and replies to other comments start at a score 5 better than the parent.

why discussion of potential LW features didn't in-advance-to-me seem very publicky

Do we really need another layer of verbal obfuscation, particularly of the cutesy variety?

Many people use the recent comments to see what is being discussed. So off topic or replies to trolls that show up there make that more difficult to use efficiently.

If you want to nuke trolling, use the Metafilter strategy: new accounts have to pay $5 (once).

I don't know if I would have made my account here if I had to pay $5 to do so. I would pay $5 now to remain a member of the community- but I've already sunk a lot of time and energy into it. I mean, $5 is less cost to me than writing a new post for main!

I am deeply reluctant to endorse any strategy that might have turned me away as a newcomer.

An unintended side-effect: readers without credit/debit cards may find it harder to join the site. This disproportionately affects younger people, a demographic that may be more open to LW ideas.

Another unintended side-effect is that it may increase phyg pattern-matching. Now new recruits have to pay to join the site, and surely that money is being secretly funneled into EY's bank account.

That said, I think that on balance this is a good policy proposal. I also think that the similar proposal using phone verification is plausible, and doesn't run into the above two problems.

I don't think anyone at SI agrees with you about Less Wrong's mission. The site is supposed to be about rationality. There is hope (and track record) of the Less Wrong rationality community having helpful spinoffs for SI's mission, but those benefits depend on it having its own independent life and mission. An open forum on rationality and a closed board for donors to a charity aren't close substitutes for one another.

we need more FAI philanthropist/activists

Who is "we"?

I think the percentage of "casual" users who participate on this site because they enjoy intelligent conversations on rationality-related topics while having no FAI agenda is non-negligible. I suspect that reinforcing the idea of equality between LW and FAI activism will make many of them leave. It may be a net negative even if LW's mission is FAI activism as there are positive externalities of greater diversity of both discussion topics and participant opinions (less boredom, more new ideas, better critical scrutiny of ideas, less danger of community evaporative cooling, greater ability to attract new readers...)

Also, I don't like the idea of LW's mission being FAI activism. There is still written in the header: "A community blog devoted to refining the art of human rationality", and I'd appreciate if I could continue believing that description. Of course I realise that the owners of the site are FAI enthusiasts, but that's not true about the community as a whole. LW is a great rationality blog even without all its FAI/philanthropy stuff, not only for the texts already written, but also for the productive debating standards used here and a lot of intelligent people around. I would regret if I had to leave, which I would if LW turned to a solely FAI activist webpage.

Casual users don't contribute to Less Wrong's mission: we need more FAI philanthropist/activists.

The tagline is still "A community blog devoted to refining the art of human rationality". If you want FAI and philanthropy, you should I suspect be asking for those specifically up front.

If you want to nuke trolling, use the Metafilter strategy: new accounts have to pay $5 (once). Troll too much, lose your account and pay $5 for a new one. Hurts a lot more than downvotes.

It's a good idea. Some variations, like associating accounts with mobile phone numbers, may slow good growth less. Maybe it would help to have multiple options to signal being a legitimate new user.

Casual users don't contribute to Less Wrong's mission: we need more FAI philanthropist/activists.

I would like to see more x-risk philanthropists/activists, but I don't want to make that a requirement for LW users. It would be good to have more users who want to be stronger because they have something to protect, rather than thinking rationality is shiny.

I think this is a VERY BAD IDEA. Charging $5 would have kept me out. It also keeps out everyone who doesn't have a credit card, which includes basically every high school student.

What percentage of current posters do you estimate are FAI philanthopists/activists?

Can you give a couple specific examples of what distinguishes them from casual users? (donates to SI? works in a relevant field? volunteers for SI? etc)

Where did you get that idea about "Less Wrong's mission" from? Actually, when LW was created, discussing AI wasn't even allowed.

Also discourages sockpuppetry

We tried to at least correct people not reading the sequence with reruns. It didn't exactly work.

I also didn't like how they fragmented the commentary. Many found that a feature rather than a bug. I found it plain annoying when reading an old article I had to do a search to see if there was any recent discussion in the rerun threads too.

I mean surely eventually some of the things we wrote back in 2007 or 2009 will turn out to have been plain wrong, obsolete or incomplete right? It would be neat to see that noted at least in their comment section.

Many people read through the sequences much like they would a textbook. We practically encourage them to do so. New well written comments to old article might be very useful.

I liked this idea until I realized I only liked it because I've been around longer than lukeprog.

Yes, tolerating tolerance is important. But also well kept gardens and all that. Indeed, part of this discussion is an attempt to try to find a way which will improve the signal to noise ratio without methods that run afoul of your concern.

I think that could severely inhibit the growth of discussion. Every new comment starts at 0 karma, and if some people didn't see those comments they wouldn't get a chance to upvote them, and they would stay at 0.

The exception to this should be when "Sort By" is set to "New".

And when it is set to "Old".

Maybe have the comment threshold increase with the age of the comment. New comments should be visible at the current -1 threshold for a day or two but after that anything below 2 could be hidden. I think that immediately hiding comments would probably lead to fewer accurate comment ratings.

Another possibility is a user-selectable threshold.

Data please: how many articles with >1000 comments are there, and what are some recent examples?

I'll also note that I just paid 5 karma to make this reply. Yet your comment was downvoted presumably because people disagreed with your suggestion, not because they thought you were trolling. The new feature of discouraging discussion on downvoted comments is counterproductive.

Explaining downvotes threatens to turn threads into metadebates about the validity of that downvote.

The names of the commentors who down-vote should be visible.

That would probably be an improvement (though there's a risk of increasing focus on meta); However -

If a person down votes a comment, it should cost them karma points to do so unless an explanation why they choose this action is made.

... would be quite bad: adding more tedious and repetitive explanations would just increase the noise, especially if the OP then answers to the explanations with even more nitpicks.

(unless you mean that downvoting would include an interface with a set of choices, but that would be cumbersome and probably incomplete)

As a neutral(neither upvoted nor downvoted) critique of that article:

The conclusion section was fairly good.

But if it was in main when you posted it, yes, it was supposed to be fully edited. I'm pretty sure drafts usually go in discussion.

And in reference to a guideline, actually yes, I think there's part of the Wiki that does mention this:

http://wiki.lesswrong.com/wiki/FAQ#I_strongly_disagree_with_Less_Wrongers_on_something._Can_I_write_a_top-level_post_about_it.3F

I strongly disagree with Less Wrongers on something. Can I write a top-level post about it?

Yes, if you do your homework first.

...

If you have a question regarding a "consensus" view, or don't want to do any homework, consider posting in an open thread.

I think you fell into the bolded section. Admittedly, Eliezer's views aren't necessarily consensus. For instance, a lot of people disagreed with him in the thread linked at the top of the page. But it probably applies in the case of the WBE vs AI idea.

Hopefully my tone came through. I'm trying to say this in a helpful/educational manner, which seems slightly silly because you've written one more article in Main then I have, so I don't know if I'm the best judge. But hopefully it helps.