Taking Good Heart Tokens Seriously, So Help Me God

post by Jackson Wagner · 2022-04-01T23:29:58.328Z · LW · GW · 4 comments

Contents

  How can we reward community involvement without creating bad incentives?
  Evaluating Impact & Better-Aligning Karma
    Important Caveat: Nuño’s “Reasons why upvotes don't correlate that well with impact .”
  Community Voting & Reputation-Sharing Across Websites
      Karma could be used as the basis for a voting system.
      Karma could be used as a pan-rationalist reputation system.
  Good Heart Tokens, (Almost) For Real
      Karma could probably somehow be used in combination with an Impact Certificate program.
  In Conclusion
None
4 comments

Summary: We probably can't just hand out a dollar per upvote and expect things to end well.  But maybe there are other, less-corrosive ways to encourage Forum contributions: 

• Maybe Karma could be used as the basis of a voting system for community decisions.
• Maybe Karma could be expanded into a pan-rationalist reputation system across multiple websites.
• Finally, maybe Karma-allocated grants could help kickstart impact certificate markets [? · GW].

How can we reward community involvement without creating bad incentives?

The idea of putting a dollar value on LessWrong and EA karma points has occurred to many people over the years. Clearly, LW / EAF / AF are producing things of value to the world, in a way that (if divided by the total number of upvotes) usually gives a value of at least a few dollars per Karma point in aggregate:

• Groups like Lightcone Infrastructure and the Center for Effective Altruism consider it worth their while to spend money and effort directly supporting these sites and encouraging their growth.
• The ongoing contributions to Effective Altruism made by hobbyists on these forums are in aggregate probably comparable in scale (in terms of how much they advance the sophistication of EA thought) to the contributions of various respected and well-funded research groups, like Rethink Priorities.
• Presumably, most of the people who read this site consider it informative and interesting. Looking at the analytics of my own top-level posts on the EA Forum [EA · GW], I get a ratio of about four upvotes per hour of reader engagement. Considering the high value of people’s time, the attention of readers suggests that posts have some value.

It would be great if we could incentivize people to produce more of this value. But of course we can’t just start handing out money in exchange for Karma, as the Good Heart Project demonstrates. It threatens to create an incentive to mindlessly churn out endless comments purely for dollars [LW · GW], or (assuming the people of LessWrong will always put in the effort to search out and destroy Good-Heart-abusing mutual-upvoting societies wherever they arise) to degrade quality in other ways by encouraging people to produce a higher volume of lower-quality drafts.

So, Good Heart Tokens as currently implemented (direct payments for newly-created karma) are probably insane. But are there other, less-crazy ideas about how to use Forum Karma to evaluate impact or reward content creation? So help me God, I honestly believe there might be a few:

Evaluating Impact & Better-Aligning Karma

Sometimes we might want to estimate the relationship between Karma and dollars even if we never intend to pay anybody anything:

• Estimating the dollar value of Forum contributions could help when evaluating impact.
  • I’m sure that philanthropists have done this when thinking about how much funding to give to Lightcone Infrastructure and the Center for Effective Altruism.
  • I myself have wondered about how to split my personal efforts between contributing to the EA Forum versus advancing other goals — should I give up commenting in favor of trying to earn a bit more money to donate to effective charities? To do the comparison, I naturally start wanting to put an imaginary dollar value on my Forum contributions.
• Thinking about how to improve Good Heart Tokens (even though the idea of Good Heart Tokens is insane) might help us spot problems and better align the regular, fake-internet-points Karma system, thus making Karma into a better reflection of what we truly value.

Important Caveat: Nuño’s “Reasons why upvotes don't correlate that well with impact .” [EA(p) · GW(p)]

Any attempt to use Karma as a proxy for value will immediately run into some problems, even if we aren’t creating Goodhart problems by paying money for upvotes. Nuño Sempere on the EA Forum lists at least 11 problems [EA(p) · GW(p)], which I will reproduce here just to emphasize that Karma is only a loose proxy for value:

1. More easily accessible content, or more introductory material gets upvoted more.
2. Material which gets shared more widely gets upvoted more.
3. Content which is more prone to bikeshedding gets upvoted more.
4. Posts which are beautifully written are more upvoted.
5. Posts written by better known authors are more upvoted (once you've seen this, you can't unsee).
6. The time at which a post is published affects how many upvotes it gets.
7. Other random factors, such as whether other strong posts are published at the same time, also affect the number of upvotes.
8. Not all projects are conducive to having a post written about them.
9. The function from value to upvotes is concave (e.g., like a logarithm or like a square root), in that a project which results in a post with a 100 upvotes is probably more than 5 times as valuable as 5 posts with 20 upvotes each. This is what you'd expect if the supply of upvotes was limited.
10. Upvotes suffer from inflation as EA forum gets populated more, so that a post which would have gathered 50 upvotes two years might gather 100 upvotes now.
11. Upvotes may not take into account the relationship between projects, or other indirect effects. For example, projects which contribute to existing agendas are probably more valuable than otherwise equal standalone projects, but this might not be obvious from the text.

It might be possible to correct for some of these factors mathematically (like #9 and #10), while other problems seem small enough that they wouldn’t be devastating for most purposes (#6 and #7 are semi-random effects that will wash out over the course of many posts), but others are quite serious.  Sometimes this would torpedo ideas completely; while for some of my ideas below it might be helpful to use an "adjusted karma-based score" that attempts to correct for a few of the problems Nuño mentions.

Community Voting & Reputation-Sharing Across Websites

Between the extremes of “completely fake internet points” versus “1 Karma = $1”, there is probably a landscape of different applications that an improved version of Karma could be used for.  The goal would be to give Karma points some additional power / influence / usefulness, without incurring as many side effects as Good Heart Tokens:

Karma could be used as the basis for a voting system.

• I could imagine having community votes about forum moderation decisions (”Should posts about Issue X be allowed on the forum?”) or about wider community issues (”Where should we try to create a second rationalist hub outside the Bay Area?”). Giving 1 vote per karma would probably too-greatly favor a small core of community members, but giving 1 vote per strong-upvote power might be useful, and would discourage people from making multiple accounts to exploit a 1-vote-per-account rule.

Karma could be used as a pan-rationalist reputation system.

• Perhaps this could be an official integration across LessWrong, metaculus and other forecasting sites, and an embeddable feature of independent blogs / substacks / etc. Making people’s reputation portable across platforms could strengthen the rationalist community by incentivizing people to produce more quality content and by increasing the reach & prestige of Rationalist/EA ideas in the wider culture.
• Alternatively to an official integration, new websites and projects like Manifold Markets could unilaterally choose to use Karma as a way to tap into an existing community — “link your LessWrong or metaculus account, and you’ll receive prediction-market play-money proportional to your reputation in those communities!”
• Or instead of linking websites, we could go in the opposite direction, creating sub-forums with sub-reputations (like the relationship of the Alignment Forum to LessWrong). For instance, perhaps the EA Forum could spawn a subforum per major cause-area, with your cred on the animal-welfare subforum determined by the amount of Karma you accrued on posts with the animal welfare tag, and so forth.

Good Heart Tokens, (Almost) For Real

How close can we fly to the “1 Karma = $1” flame?  Maybe pretty close:

Karma could be used to assign vote power for a quadratic-funding system devoted to Rationalist / EA community-building efforts, and other community public goods. With this system, users could quadratically vote on how much funding should be directed to different goals: Forum website improvements, or hosting conferences, or providing in-person services like childcare to bay-area rationalists, or beefing up the community’s ability to support independent researchers [EA · GW], or etc.

Under this system, you could use your LessWrong karma points to influence a pot of public-goods funding towards projects that benefit you (like in-person Bay-Area services if you live in the Bay). But this is a pretty indirect effect, and the self-serving financial incentive is laundered through a process of feel-good community participation and broad-based public-goods production, so I think it would be a lot less dangerous than literal Good Heart Tokens.

Karma could probably somehow be used in combination with an Impact Certificate program.

The current idea behind Impact Certificates [? · GW] looks something like this:

1. An organization like OpenPhil pledges to buy $1,000,000 of “impact certificates” from impactcerts.com in 2025. They’ll buy certificates according to their estimate of the impact each project had.
2. Today in 2022, effective altruists seeking funding for their projects and research can mint “impact certificates” describing what they’ll do, and sell them to speculators.
3. Between now and 2025, speculators will trade the impact certificates, attempting to guess how OpenPhil will eventually value the outcome of the projects linked to the certificates. They’ll be profit-motivated to act as an army of decentralized grant evaluators, seeking out and assessing overlooked projects that OpenPhil would love to fund.

In order to kick-start this market, it might be useful to airdrop impact-certs credit to people as a function of their EA & LessWrong forum reputation — you get free money, but you have to use it to buy impact certificates (which you can either hold forever as a form of charitable donation, or sell immediately at a low price to cash out, or wait until 2025 in the hopes of being bought out by OpenPhil for a higher price).  Providing the initial airdrop could help in two ways: it would create lots of initial trading interest (getting a bunch of smart people looking at the certificates and doing grant evaluations), and it would help create a new cultural norm that holding impact certificates is a cool way of donating to charity, showcasing your values, and supporting the rationalist community.

With an impact-certs airdrop, there would be a danger that many people immediately cash out and don't engage with the system, although it might be possible to disincentive this somehow.  Even if not, I think the danger of creating bad incentives would still be much lower than with Good Heart Tokens -- perhaps even low enough that it would be a good idea -- precisely because the money would be given out as a one-time grant based on people's retrospective community contributions.  Any incentive to start churning out trash on the forum would be very weak, since it would have to be based only in the vague hope that a future project might someday do a similar airdrop.

In Conclusion

I think there are some legit ways that karma could possibly be used as part of interesting experiments in decentralized grantmaking and community decisionmaking.  Despite the fact that the Good Heart Project is hilarious and obviously insane, the promise of sweet, sweet $1 upvotes was the very thing that motivated me to write and publish this post, so I would like to thank the organizers of the Good Heart Project for doing all the work necessary to run this wacky but intensely thought-provoking experiment.

4 comments

Comments sorted by top scores.