EigenKarma: trust at scale

I don't like that this conflates upvoting someone's writing with trusting their voting judgement.  The VAST majority of upvotes on most systems comes from people who don't post (or at least don't post much), and a lot of posters who I like their posts, I disagree pretty strongly with their liking on other topics.  

More importantly, I think this puts too much weight on a pretty lightweight mechanism, effectively accelerating the goodhart cycle by making karma important enough to be worth gaming.

A guess: if LessWrong implemented this, onboarding lots of new users at once would be easier to do without ruining the culture for the people already here. 

I think a problem with this is that it removes the common-knowledge-building effect of public overall karma, since it becomes much less clear what things in general the community is paying attention to.

I am not sure if the unified "trust", let alone "transitive trust" makes sense. People can be experts on something, and uninformed about something else. There are people I trust in the sense "they wouldn't stab me in the back", but I do not trust their trust in homeopathics or Jesus. In context of LessWrong, I would hate to see my upvotes of someone's articles on math translated as my indirect support of Buddhism.

I don't think this quite works, but I like the attempt. The problem I see here is that this is likely to create filter bubbles. One of my strategies for avoiding filter bubbles is that I often specifically seek out media that is unpopular and then simply try to get through a lot of it fast, because it is rare for what I want to correlate terribly strongly with what others want. Also, upvoting someone's comments doesn't mean that I agree with them, and agreeing with them doesn't mean that I trust them to recognize what's good in the same situations I would. I would suggest that a key problem with karma is in fact the issue that there's a single direction of up/down, but I think there's something more fundamentally funky about the idea of having a "upvote so others can see" view, even as it exists now. I'd personally suggest that votes should be at the same level as comments - votes should be seen as reviews, in the same sense as scientific reviews. And even scientific review has serious problems. index of last time I did a search for this. 

In general, I think what we'd want would have some degree of intentional partitioning as new nodes get added, and some degree of intentional anti-partitioning; the graph should probably be near the edge of criticality in some key aspect, as most highly effective systems turn out to be, but figuring out which feature should be edge of criticality is left an open question by that claim.

It might make sense to separate simulacrum 1, 2, and 3 - fact, manipulation, and belonging -  intentionally, if possible; getting them to stay separated, or to start out separated even for a new user, is not trivial. How could something like EigenKarma be adapted to do this? Dunno.

I created a market on Manifold about whether the EA Forum or LW will start using EigenKarma by 2025.

There have been experiments with attack-resistant trust metrics before. One notable project was Advogato. It failed and I'm not sure why. It's archived now. Maybe because it didn't create individual graphs. It might be worthwhile to look into Advogato's Trust Metric.

To my best understanding this is basically doing PageRank but with votes taking the place of links - so a user's outgoing trust is divided between other users in proportion to how much in total they've upvoted each one.

I could well be wrong though, the documents include a low-level description of the algorithm if people want to check.

It seems to me this approach would be likely to strongly favor more prolific users, and I would guess that, even if outsiders didn't agree with the core of prolific users, they would tend to see results weighted heavily towards those users and whoever those users most upvote.

I would much prefer an approach that compensated for this in some way.

This seems similar to Personalized PageRank, which is widely used in recommendation systems.

What’s great about the current setup is it doesn’t produce any additional friction for users since it’s built on top of an existing post-voting system. However, one downside is that number of my given upvotes doesn’t perfectly tracks the trust I assign to the user. E.g. as pointed out by previous commenters, prolific posters could post more often and collect more likes than they intuitively “deserve.” This particular issue can be mitigated by capping the effective number of upvotes to a given user by certain defined number.

The most popular Russian speaking poker forum uses a karma system, where everyone can explicitly assign karma to each other in a range [-100:100]. It served as an analog of google map reviews for users, and I personally found it super useful, since it helped distinguish trustworthy people and helped establishing financial relations (like lending or staking each other). I should say it worked pretty well! 
I think displaying the eigenvector of this karma matrix, would be even more useful. EigenKarma would also allow making anonymous ratings while being resistant to botnets.

I imagine this would be hard to sell (in the sense of getting them to let you hook into their votes and calculate karma and use it to determine what users see) to companies like Facebook and Twitter that show you lots of content that you can vote on. My guess is because they want control over what people see so they can optimize it for things they care about, like generating ad revenue or engagement. For many sites, what the user wants is just an input to consider; the site is optimizing for other things that may not reflect the users preferences but that's okay so long as more the desired objective is obtained, like placing ads that people click on.

There is one concern about the transitive nature of trust:

Emmett Shear:

Flattening the multidimensional nature of trust is a manifestion of the halo/horns effect, and does not serve you.

There are people I trust deeply (to have my back in a conflict) who I trust not at all (to show up on time for a movie). And vice versa.

Paul Graham:

There's a special case of this principle that's particularly important to understand: if you trust x and x trusts y, that doesn't mean you can trust y. (Because although trustworthy, x might not be a good judge of character.)

https://mobile.twitter.com/paulg/status/1627843923811991552 

FYI, eigenkarma's been proposed for LessWrong multiple times (with issues supposedly found); see https://www.lesswrong.com/posts/xN2sHnLupWe4Tn5we/improving-on-the-karma-system#Eigenkarma [LW(p) · GW(p)] for example.

I suspect this is less accurate at recommending personalized content compared social media algorithms (like tiktok) that consider more data, yet is also not much more transparent than those algorithms.

You could show the actual eigenkarma - but you'd have to accurately convey what that number means, make sure that users don't think it's global like reddit/hn, and you can't show it when logged, in link previews, nor in google search. Compare this to the simplicity of showing global karma - it's just a number and 2 tiny buttons that can be inline with the text. LW jams two karmas in each comment and it makes sense.  The anime search website Anilist lets users vote on category/genre tags and similar shows on each show's page, and it all fits.

I think "stuff liked by writers who wrote stuff I like" is less accurate than "stuff liked by people who liked content I like". There are usually much fewer writers than likers.
I think it's also less transparent than "stuff written by writers I subscribed to"

A desideratum that comes to mind for such a system is "Reallocating 10% of my direct trust changes my indirect trust in Alice by at most 20%." - does something like that hold? What desiderata characterize your algorithm?

This is kinda like the "liquid democracy" software, but with delegation being automatic instead of, say, thoughtful. I like the idea of seeing only some people's upvotes, but I want to consciously and explicitly choose those people. Not by subscribing to their posts, and definitely not by upvoting something they've written.

I could imagine a toggle-switch next to the upvote counter, labeled "upvotes you trust, only" or something, along with a little link to go whitelist users into your trust-graph.

A related-but-not-the-same idea, would be to add a lot of different special upvotes to posts and comments. I think the max (before it becomes dumb) is like 5-6. We have "normal upvotes" and "agree/disagree" (but only on comments, for some reason???). If we replaced "normal upvotes" with another axis, we'd get 4-5 new axes to play with.

Axes might include "I think people outside our community should hear this", "I think this is novel in a useful way", "I personally made life-changing decisions based on this article, and now N years later I feel positive/negative about them" (this one should be locked behind a timer of some sort), "I predict this article will help me with decisions" (see previous), "this post's existence decreased X- (or S-) risks", "number of markets that linked this post on Manifold", "I liked the emotional vibe (or writing style or...) of this post", the aforementioned whitelist-eigenkarma, etc.

The upvote numbers would all be equally-sized (small), and arranged in a cute ring, kinda like how some RPGs/cardgames display stats. Maybe the user could choose one from the ring as their "default score shown" (which would also inform what posts are shown to them).

As a community acutely aware of platform dynamics, we should really be doing a better job of this.

Half the "inherent platform problems karma upvotes reddit digg dynamics doom" may be surprisingly solvable with UI tweaks. Especially UI that doesn't do certain things by default. (E.g. you only see "one karma number" if you opt-in and pick which karma-axis you want; you only see "eigenkarma" based people you've explicitly whitelisted; etc.).

If I get a lot of Karma in the area of Set Theory, but I give a lot of Karma in the area of blockchain technologies, should this Karma be worth as much as the Karma I give in Set Theory? I do not think so.

Can there be a mechanism that boosts posters who get upvotes from multiple nonoverlapping groups extra? If eg 50 Blues and 50 Greens upvote someone, I want them to get more implicit eigenkarma from me than someone with 100 Blue upvotes even if I tend to upvote Blue more often. Figuring out who is a Blue and who is a Green can be done by finding dense subgraphs..

The LessWrong Review [? · GW] runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2024. The top fifty or so posts are featured prominently on the site throughout the year.

Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?

Are you familiar with Constellation's Proof of Reputable Observation? This seems very similar.

Cool idea. I created an account (MYZ) but  I'm hesitant to use it because there seems to be only a binary choice between 'Trust' or not. 

Could there be a slider instead? i.e. for some intermediary level of trust.