Karma awards for proofreaders of the Less Wrong Sequences ebook
post by alexvermeer · 2013-12-12T00:18:53.375Z · LW · GW · Legacy · 59 commentsContents
LW Karma Bonus None 59 comments
MIRI is gathering a bunch of Eliezer’s writings into a nicely-edited ebook, currently titled The Hard Part is Actually Changing Your Mind. This book will ultimately be released in various digital formats (Kindle MOBI, EPUB, and PDF). Much of the initial work for this project is complete. What we need now are volunteers to review the book's chapters to:
- verify that all the content has been correctly transferred (text, equations, and images),
- proofread for any typographical errors (spelling, punctuation, layout, etc.),
- verify all internal and external links,
- and more.
This project has been added to Youtopia, MIRI’s volunteer system. (Click “Register as a Volunteer” here to sign up. Already signed up? Go here.)
LW Karma Bonus
For this special project, every point earned in Youtopia will also earn you 3 karma on LW!
Points are awarded based on the amount of time spent proofreading the book. For example, an hour of work logged in Youtopia earns you 10 points, which will also get you 30 LW karma. Karma is awarded by admins in a publicly-accountable way: all manual karma additions are listed here.
Questions about this project can be directed to alexv@intelligence.org or in the comments.
59 comments
Comments sorted by top scores.
comment by Shmi (shminux) · 2013-12-12T17:11:48.543Z · LW(p) · GW(p)
Points are awarded based on the amount of time spent proofreading the book.
What, no bug bounty?
comment by Viliam_Bur · 2013-12-20T15:18:18.540Z · LW(p) · GW(p)
Just wanted to say: Whoever updated the texts from the original web version to the version used in the book, they did a very good job. So far I have compared only one chapter, but I am impressed. The changes are typically rephrasing a part of a sentence, or reordering some paragraphs, but the new version feels better. I am happy to have a more polished version of the Sequences, in addition to just another format.
(I am not proofreading. I translated some of the original articles to Slovak, and now I am updating my translations using the version from the book, with the goal of making a translation of the whole book one day. This is why I automatically noticed the differences between the original and the new versions.)
comment by polymathwannabe · 2013-12-18T00:20:19.926Z · LW(p) · GW(p)
Proofreading is my calling and my reason to live. Count me in.
comment by Viliam_Bur · 2013-12-20T21:15:16.612Z · LW(p) · GW(p)
The book is not a blog; it is not published step by step, but as a whole, at the same time.
Therefore, I suggest removing comments like "Continued in next post" or "Followup to"; especially if it is a reference to the very next or very previous chapter, but also if it is two chapters behind. Specifically, chapters 3 and 4, 11 and 13, 20 and 22, 22 and 23, 28 and 29, 31 and 32, 31+32 and 34, 41 and 42, 49 and 50, 54 and 55, 56 and 57, 57 and 58, 59 and 60, 60 and 61, 62 and 63, 63 and 64, 64 and 65, 64 and 66... okay, I guess you get the idea. In a book, the next chapter being a followup to the previous chapter is pretty much expected.
On the other hand, chapters 16, 18, 28... are followups to chapters not included in the book. Maybe instead of "followup" it would be better to use some other words to express that this chapter is related to some online article.
Summary: the book is a different medium, so I suggest rewriting the standard "followup" notice, depending on situation:
- it it's a followup to the previous chapter (maybe also two chapter behind), just remove it;
- if it's a followup more chapters behind, use the full sentence to explain it;
- if it's a followup to something not included in the book, again use the full sentence, but a different one.
comment by Brillyant · 2013-12-13T19:24:22.989Z · LW(p) · GW(p)
I think this might be another way the karma system continues to be a feature of a map that doesn't necessarily represent the territory.
High karma seems to me to be about being on board with LW orthodoxy, articulating that clearly in posts and comments and supporting the aims of the group.
I guess there is nothing wrong with that...though that is exactly how "karma points" are handed out in the church.
This is basically saying, "People will listen to your opinion X amount more inside LW for doing this volunteer work." It is the exact same way in the church for those who evangelize a lot or display some other evidence of their commitment to the group.
The problem, I think, is that proofreading e-books or evangelizing the group's message is not the slightest bit related to having a view that is in line with reality which is, as I understood it, one of the main aims of rationality.
Replies from: Cyan↑ comment by Cyan · 2013-12-14T03:14:07.709Z · LW(p) · GW(p)
High karma seems to me to be about being on board with LW orthodoxy, articulating that clearly in posts and comments and supporting the aims of the group.
The criticism you proffer has been made independently several times. The usual answer is that cogent, intelligent criticism that shows familiarity with the premises and conclusions of LW "orthodoxy" (and clearly locates its criticism as a problem with either the plausibility of a premise or the validity of an argument, or both) tends to get upvoted. You just have to show you've done your homework.
Replies from: Brillyant↑ comment by Brillyant · 2013-12-14T18:01:05.540Z · LW(p) · GW(p)
The problem is that exactly what qualifies as "homework" is determined by the in-group. And, as I said, this is exactly how it works in the church.
Nevermind that though. My point was really that karma isn't tied to who is right, it is tied to who we like or who furthers our preferential ends. This karma-for-work deal is another example of that.
If karma is a popularity system, then fine. But there seems to be a lingering sentiment it is more about rationality, and how a given comment or commenter is in line with it. That's not the case when you are giving people points to do tasks.
Replies from: Nornagest, Kaj_Sotala, Cyan↑ comment by Nornagest · 2013-12-16T18:08:03.166Z · LW(p) · GW(p)
If we feel that we want to keep track of instrumentally useful contributions to the community but also want karma to remain a more-or-less pure representation of the reception of a user's comments, then the obvious way to reflect that seems to be to create a metric other than karma to represent the former. This might also be a useful way of reflecting certain actions that have traditionally been rewarded by means of upvotes on comments bragging about them, like posting survey responses or donating to CFAR or MIRI. Granted, the development resources for this aren't likely to appear in the near term.
Elsewhere in these comments I've mentioned the XP metric that the Everything2 community created to fill a karma-like niche. It was after my time, but I'm told they ended up creating a "GP" metric (RPG metaphor, yes) that worked similarly to this.
Replies from: Viliam_Bur↑ comment by Viliam_Bur · 2013-12-20T15:11:38.308Z · LW(p) · GW(p)
create a metric other than karma to represent the former
badges
↑ comment by Kaj_Sotala · 2013-12-16T09:53:20.772Z · LW(p) · GW(p)
My point was really that karma isn't tied to who is right, it is tied to who we like or who furthers our preferential ends.
Barring an objective method for telling what arguments are right, this is the way any human-run evaluation system (including e.g. formal peer review or university grades) has to work. You can try to eliminate the "who we like" part by trying to blind the identities of the people in question, but since one cannot assess degree-of-correctness directly, one has to rely on some other criteria, e.g. the extent to which the comment seems to take previous work into account. And those other criteria and their parameters, like what counts as "previous work", are ultimately set by an in-group consensus. (I felt that James Paul Gee had a particularly good elaboration of this.)
But I do agree that awarding karma for work distances karma from correctness even further than would be necessary. (Not sure whether it's a bad thing, though.)
↑ comment by Cyan · 2013-12-15T02:27:48.191Z · LW(p) · GW(p)
At the very low end, a certain amount of karma must be earned for certain functionality to become available to a user account. But past a certain point -- somewhere in the 200-500 range -- more karma just doesn't matter very much when assessing a user account. After that point, the meaning of a karma score attaches much more strongly to specific comments than to users.
The corollary is that the karma reward will incent newcomers a lot more than old hands. I don't have a problem with that.
My point was really that karma isn't tied to who is right, it is tied to who we like or who furthers our preferential ends.
The "who we like" part seems to be how some people operate, especially vis-a-vis mass downvoting. I view that as counterproductive. What the phrase "furthers our preferential ends" denotes is not clear to me; if it means something like an upvote is meant to signal "I want to see more like this comment" and a downvote the opposite, then my understanding of LW consensus is, yes, that's the idea.
Replies from: Brillyant↑ comment by Brillyant · 2013-12-15T16:16:32.000Z · LW(p) · GW(p)
But past a certain point -- somewhere in the 200-500 range -- more karma just doesn't matter very much when assessing a user account.
Huh? If karma isn't very meaningful past a certain count, why keep track of it at all? Why not just call everybody who reached 500 karma points "vetted" and leave it at that? (I suspect the answer is that karma does matter to some significant portion of the people here, but I'm open to hearing why you think otherwise.)
The "who we like" part seems to be how some people operate, especially vis-a-vis mass downvoting. I view that as counterproductive. What the phrase "furthers our preferential ends" denotes is not clear to me
"Who we like" could include mass downvoting. More than it, it involves applause lights hanging above certain members who espouse popular views or who have done something, apart from making a rational comment, to garner favor in the group. This could be proofreading a LW-approved text, organizing a meetup, etc., etc.
The key characteristic of this earned karma is that is has zero to do making a direct contribution to a more accurate map.
It's is just interesting to me because it's a lot like the church from whence I came. Signaling devotion to the cause becomes more important than being right about the merits of the cause.
LW is a community specifically committed to the mission of "refining rationality" and, therefore, (you'd think) making sure things like karma systems work to incentivize members toward that end. I don't see that happening.
LW is the best blog/forum I've seen -- virtually troll free, consistent flow of interesting articles, thoughtful & well-written comments, lots of people waaaay smarter than me, etc. That is part of why it is so interesting that the karma system seems so...weak.
Replies from: Cyan, Kaj_Sotala↑ comment by Cyan · 2013-12-16T12:12:26.914Z · LW(p) · GW(p)
Huh? If karma isn't very meaningful past a certain count, why keep track of it at all? Why not just call everybody who reached 500 karma points "vetted" and leave it at that? (I suspect the answer is that karma does matter to some significant portion of the people here, but I'm open to hearing why you think otherwise.)
I didn't say it was strictly meaningless or negligibly meaningful.
The reasons for not having an explicit "vetted" status in lieu of accumulating karma are to a certain extent historical -- it wasn't thought of when the LW karma system was implemented, since that was adapted whole-hog from the Reddit codebase. I think the reason why such a change hasn't been made in the meantime is three-fold: (i) it would obviate the "Top Contributor, 30 Days" status incentive (the most feasible way to top that list is to write highly upvoted front page articles), (ii) it would obviate any loss-aversion-motivated engagement induced by each account's "karma in the last 30 days" score; and (iii) on general "if it ain't broke don't fix it" grounds.
It's is just interesting to me because it's a lot like the church from whence I came. Signaling devotion to the cause becomes more important than being right about the merits of the cause.
Yes, churches win at creating socially cohesive communities. If your complaint is that that karma system induces undue (in your view) social cohesiveness, my response is, "feature, not bug". (Here's some LW canon on the usefulness and pitfalls of social cohesiveness.) If you suspect that the harmfulness of the system outweighs the usefulness, set some standards for harmfulness and usefulness and then collect some evidence for and against that hypothesis. Keep in mind that by design, the biggest rewards go to high-quality front page posts (like this one criticizing time spent kibitzing on LW).
Replies from: Brillyant↑ comment by Brillyant · 2013-12-16T15:16:37.305Z · LW(p) · GW(p)
I didn't say it was strictly meaningless or negligibly meaningful.
Nor did I say you said that. You said this:
But past a certain point -- somewhere in the 200-500 range -- more karma just doesn't matter very much when assessing a user account.
And I replied with this:
If karma isn't very meaningful past a certain count, why keep track of it at all?
My apologies if you feel I rephrased you inaccurately, or missed your meaning.
The reasons for not having an explicit "vetted" status in lieu of accumulating karma are to a certain extent historical -- it wasn't thought of when the LW karma system was implemented, since that was adapted whole-hog from the Reddit codebase. I think the reason why such a change hasn't been made in the meantime is three-fold: (i) it would obviate the "Top Contributor, 30 Days" status incentive (the most feasible way to top that list is to write highly upvoted front page articles), (ii) it would obviate any loss-aversion-motivated engagement induced by each account's "karma in the last 30 days" score; and (iii) on general "if it ain't broke don't fix it" grounds.
To the system being historical: "That's the way we've always done it" isn't a very good reason for any policy or behavior to continue.
To (i) and (ii): Keep 30-day karma and ditch cumulative karma. No need to obviate anything.
To (iii): Begging the question.
Yes, churches win at creating socially cohesive communities. If your complaint is that that karma system induces undue (in your view) social cohesiveness, my response is, "feature, not bug".
Social cohesive is fine, of course. I agree it is a feature. And it is great!
It (social cohesiveness) shouldn't take priority over adherence to reality. When it does, it's buggy. And it is what happens in the church. In my experience, they value "unity" over rationality. At LW, that is a clear no-no. (I agree it is a no-no. Make sure you are correct first; only then be unified. Lest dogma tends to ensue.)
If you suspect that the harmfulness of the system outweighs the usefulness, set some standards for harmfulness and usefulness and then collect some evidence for and against that hypothesis.
This is way outside my level of interest in, or commitment to, LW. I've given what I believe to be a reasonable criticism of the karma system (one that you mention has been independently noted many times) and made what I believe to be an accurate and helpful analogy (i.e. how "karma" works in the church).
It's on the record for anyone who is interested to do with it what they'd like, or ignore it altogether.
Keep in mind that by design, the biggest rewards go to high-quality front page posts (like this one criticizing time spent kibitzing on LW).
One of my favorite posts. Though I would say reading and interacting on LW is really good for rationality novices -- like me.
At some point, it does become a bit of an anti-rational engagment. For me, in this thread, that time is now.
Tap.
Replies from: Lumifer, Cyan↑ comment by Lumifer · 2013-12-16T16:02:20.041Z · LW(p) · GW(p)
Keep 30-day karma and ditch cumulative karma.
So people who don't post on LW for a month or more become indistinguishable from newbies?
Replies from: ialdabaoth↑ comment by ialdabaoth · 2013-12-16T17:44:09.711Z · LW(p) · GW(p)
What happens if old karma is only displayed as a percentage, rather than as a number? That way you know generally what the community thinks of their post quality, without more-prolific posters overwhelming less frequent posters?
Replies from: Lumifer↑ comment by Kaj_Sotala · 2013-12-16T09:59:38.681Z · LW(p) · GW(p)
LW is the best blog/forum I've seen -- virtually troll free, consistent flow of interesting articles, thoughtful & well-written comments, lots of people waaaay smarter than me, etc. That is part of why it is so interesting that the karma system seems so…weak.
Are you sure that the other features you mention aren't in part because of the "weak" karma system? Trolls get actively downvoted, for instance.
Replies from: Brillyantcomment by passive_fist · 2013-12-12T21:13:59.704Z · LW(p) · GW(p)
Will the ebook be offered for free?
Replies from: alexvermeer↑ comment by alexvermeer · 2013-12-13T23:58:56.122Z · LW(p) · GW(p)
It will probably be available as pay-what-you-want, similar to the Facing the Intelligence Explosion ebook.
comment by [deleted] · 2013-12-15T16:00:45.232Z · LW(p) · GW(p)
The LW Karma idea makes me uncomfortable. And I'm saying this as a user who doesn't have that much karma, and might be interested in doing this.
comment by brazil84 · 2013-12-14T20:12:40.818Z · LW(p) · GW(p)
If you come by my office and shine my shoes, I will upvote your posts until you get an extra 500 karma points. Doing my laundry will net you a cool thousand karma points.
Replies from: Nornagest↑ comment by Nornagest · 2013-12-14T21:02:15.469Z · LW(p) · GW(p)
I think I'd rather do my own laundry than upvote a thousand posts. If the average user even has a thousand posts.
I suppose that's what scripts are for, though.
Replies from: ialdabaoth↑ comment by ialdabaoth · 2013-12-14T21:05:23.542Z · LW(p) · GW(p)
Indeed, and they aren't too hard to find and modify.
EDIT: Hi there! Since you're downvoting everything I say at this very minute, and haven't got to this post yet, could you please stop for a second when you do and say 'hi'?
EDIT2: Guess not.
Replies from: shminux↑ comment by Shmi (shminux) · 2013-12-15T07:43:09.780Z · LW(p) · GW(p)
don't feed.
comment by ChristianKl · 2014-10-23T21:28:58.253Z · LW(p) · GW(p)
What's the current estimated time when the project will finish?
Replies from: alexvermeer↑ comment by alexvermeer · 2014-10-25T22:11:31.402Z · LW(p) · GW(p)
Hopefully by the end of the year.
comment by Ixiel · 2014-10-24T19:37:59.967Z · LW(p) · GW(p)
Is this going to be released as a real book as well? I see only mentions of digital release in the post.
Replies from: alexvermeer↑ comment by alexvermeer · 2014-10-25T22:11:46.676Z · LW(p) · GW(p)
The plan is to do both.
Replies from: Ixielcomment by TheTerribleTrivium · 2013-12-12T17:34:45.123Z · LW(p) · GW(p)
I recently read the sequences at a fairly rapid pace using the ebooks versions available on my ipad and there were certainly a few common things I found intensely irritating. I definitely found some of the sequences interesting enough to read again more closely. Which of the sequences are included in the ebook?
Replies from: alexvermeer, kalium, ThrustVectoring↑ comment by alexvermeer · 2013-12-14T00:01:42.582Z · LW(p) · GW(p)
Basically all of them, with some modifications (e.g. a significantly reduced QM sequence), and with some reorganizing to improve flow.
Replies from: Halfwitz↑ comment by kalium · 2013-12-14T01:19:05.269Z · LW(p) · GW(p)
Which particular things did you find irritating?
Replies from: TheTerribleTrivium↑ comment by TheTerribleTrivium · 2013-12-14T11:05:17.075Z · LW(p) · GW(p)
Poor handling of images, maths sections and inter-sequence links were the most notable ones I remember
↑ comment by ThrustVectoring · 2013-12-14T15:12:26.633Z · LW(p) · GW(p)
I've proofread three of the chapters. There are 340 of them in the version of the ebook that I was sent, and the whole thing runs over 2000 pages. At roughly 250 words per page, that's 500k words.
So, basically what alexvermeer said - pretty much all of them.
Replies from: BlueSun↑ comment by BlueSun · 2013-12-23T20:52:06.006Z · LW(p) · GW(p)
Does anyone know what happened to the version that was supposed to be reviewed/edited down by a professional so it could be publishable length? There's so much good stuff there I'd love to be able to send to friend and family but 500k worth of blog posts is much harder to send someone than a nicely published 200k version.
comment by V_V · 2013-12-12T18:56:47.713Z · LW(p) · GW(p)
Quiz: what kind of organizations solicit hard work from their members offering only an immaterial reward as compensation?
For example, an hour of work logged in Youtopia earns you 10 points, which will also get you 30 LW karma.
Ah ok, keeping your browser logged in a website while attending to your own business is not hard work...
Replies from: Nornagest, TheOtherDave, asr, None↑ comment by Nornagest · 2013-12-12T19:09:03.333Z · LW(p) · GW(p)
what kind of organizations solicit hard work from their members offering only an immaterial reward as compensation?
Wikipedia, YouTube, Reddit, any blogging service...
Replies from: V_V↑ comment by V_V · 2013-12-12T19:20:04.529Z · LW(p) · GW(p)
YouTube actually pays with real cash people who upload popular videos. AFAIK the other organizations you mentioned don't give away "karma points" in exchange of work.
Replies from: Nornagest, gwern↑ comment by Nornagest · 2013-12-12T19:23:42.405Z · LW(p) · GW(p)
The point is that using social status to motivate people who provide content or do other useful work is very common; it's arguably one of the defining features of what the press calls Web 2.0. Karma is just a (crude) way of quantifying that.
I first saw a similar quantification scheme on Everything2, an early competitor to Wikipedia that morphed into more of a writers' platform when it became clear that Wikipedia was winning. They called it XP, though.
Replies from: V_V↑ comment by V_V · 2013-12-13T14:10:10.700Z · LW(p) · GW(p)
IIUC these reputation systems are based on user votes.
LessWrong karma is usually also generated that way, at least until now. But if MIRI starts to sell LessWrong karma in exchange of work, then what's the point of having a karma system? What does 'karma' represent?
↑ comment by TheOtherDave · 2013-12-13T16:02:15.328Z · LW(p) · GW(p)
What did it represent until now?
If your answer is "user votes," as your comment suggests, then I guess what it represents now is "a combination of user votes and MIRI work." Is the latter less valuable a thing to represent than the former?
↑ comment by TheOtherDave · 2013-12-12T20:36:37.223Z · LW(p) · GW(p)
what kind of organizations solicit hard work from their members offering only an immaterial reward as compensation?
volunteer organizations
Replies from: V_V↑ comment by V_V · 2013-12-13T14:13:01.718Z · LW(p) · GW(p)
Do they award "karma points"?
The closest example I can think of is the Catholic Church selling indulgences.
Replies from: TheOtherDave↑ comment by TheOtherDave · 2013-12-13T15:54:30.863Z · LW(p) · GW(p)
Volunteer organizations motivate and compensate their members in a variety of ways. Most of the ones I'm acquainted with do so via immaterial rewards, which was your original question.
With respect to the new goalpost of "karma points" -- can you clarify what characterizes the set of "karma point" like reward structures? If the Catholic Church selling indulgences qualifies, I'm not sure why wikipedia granting wider edit privileges wouldn't, for example... those things are certainly different from one another, but then again both are different from karma points. I'm not sure what differences are significant, on your view.
↑ comment by asr · 2013-12-13T14:30:20.121Z · LW(p) · GW(p)
Quiz: what kind of organizations solicit hard work from their members offering only an immaterial reward as compensation?
Lots. Scientific publications solicit hours of work from reviewers, for every paper published, without compensation or even much in the way of professional credit to the anonymous reviewers.
Replies from: V_V↑ comment by V_V · 2013-12-13T20:06:31.174Z · LW(p) · GW(p)
so they don't offer any reward. My point.
Replies from: asr↑ comment by asr · 2013-12-13T21:46:30.233Z · LW(p) · GW(p)
Hrm? As a scientific reviewer, you get the reward of being able to list "reviewer for XYZ" on your CV. Which is an "immaterial reward", probably about on a par with LW karma. Both are a mild point of prestige, but not really enough to boast about in isolation.
Replies from: IlyaShpitser, shminux, V_V↑ comment by IlyaShpitser · 2013-12-14T17:41:39.964Z · LW(p) · GW(p)
No one cares about that. The real point of reviewing papers is keeping up w/ your field.
↑ comment by Shmi (shminux) · 2013-12-15T07:46:00.145Z · LW(p) · GW(p)
No one ever lists that in any field I know.
↑ comment by V_V · 2013-12-14T16:52:58.551Z · LW(p) · GW(p)
So, if reviewers are allowed to mention the fact on their CV, and this provides a significant, even if small, professional advantage, then that's a real reward in the sense that it leads to material benefits.
I doubt that LW karma is of much help in that direction.