Tag Relevance Systems (Feedback Requested)

Here's what I consider the relative merits of each Tag Relevance system. (Note: it'd be helpful if you read the OP and form your own opinion before reading this)

...

...

Karma-Style 

Advantages:

• It re-uses an existing system, which makes it easier to program, and easier to learn.
• It's very easy to understand how the sorting works – it very transparently sorts in descending relevance order.

Disadvantages:

• It's hard to move a recently-tagged post to the bottom of the list without removing it entirely, or upvoting every other post instead. 
• I personally find that it messes with my intuitions about how karma works. I find myself wanting to upvote posts I like in the usual way... but I already did that. While re-using an existing system is mechanically simpler, I personally find it requires more conceptual orientation.
• My votes are often not about which posts are objectively relevant, but about which posts I think are more relevant than the current consensus suggests. This a) contributes to voting feeling unintuitive to me, and b) since my votes are very "current-context dependent", I worry that later I'll have to go and re-vote on things to make sure my vote is still doing what I want.
• It's fairly overwhelming for the tag page to list both "relevance" and "karma", but if you only show one of those, it's confusing in a different way (if you leave off karma, people may mistakenly think relevance is karma. If you leave off relevance, people will be confused about how the page is sorted)
• A single user can't set up an entire tag page in a reasonable fashion – if you're adding a new post, you basically only have two choices, "smallVote" and "bigVote", so if you want to add anything that's only tangentially relevant, you have to have it mixed in with all the other smallVotes until another user comes along to flesh things out.
• When only a couple people have voted on a Tag Page's posts, I think the results are worse than just sorting by karma.

I expect many of these problems to mostly effect less-visited tags. More visited tags will probably end up with lots of users contributing, resulting in a "wisdom of crowds". But I expect at least half of Tag Pages to be sparsely attended, and for the relevance votes to be much noisier than just sorting by karma.

Multiple Choice Voting Style

Advantages:

• A single user can vote honestly based on their true beliefs about how relevant a tag is, adding a bunch of tags at once and accurately sorting them.
• I personally find it more intuitive (I don't have any conflicting intuitions ported over from karma)
• The Tag Page becomes simpler – you don't need to show relevance at all, you just use it to establish major categories.
• It lets karma remain relevant, without being very confusing. (A few people have suggested "sort by relevance + karma", but the problem is that makes relevance much more confusing, removing it's clarity advantage)

Disadvantages:

• It requires building a whole new system. This is nontrivial. (It's probably only around a week of work, but there are lots of other things we could build in a week, and if most people find Karma Relevance pretty intuitive it may not be worth it)
• Users have to learn a new system. I'm guessing that system wouldn't be that hard to learn, but I think "median vote" is a somewhat more confusing concept than "sort by a number" (especially when you factor in vote weight)

I found the current karma-style relevance to be surprisingly intuitive. My impression of tags is that the main problem is figuring out what is allowed to be a tag or not - I'm not sure how this was done, but the tags seem to be good.

I think "top" posts can be some combination of high karma, high tag relevance, and curated.

There is the additional problem of "Meditations on Moloch" was read by a lot of people, so it will get high karma and high tag relevance, and maybe be curated because it is high quality. This suggests that the multiple choice system is better able to bucket posts.

I think I want any tagging system to be able to separate between posts that use the concept and posts that try and explain the concept. There will not be that many of the later, but they should be prioritized on the tag's page (there are possibly only 1 or 2, in which case a form of curation but for specific tags might solve the issue).

I like the current karma style relevance system and think it's worthy to be given a good trial period. If after a year we find that not enough people vote on tags, we can introduce other features, but I think the tag functionality as it is functional enough to warrant a good trial period.

Meta on format: Why didn't you use the question feature for this post?

Meta on content: yay, I really like those kind of improvement!

Object: epistemic status: thinking outloud right after having read the post, before reading the comments (as requested), and not having thought about this much before; not confident

it seems like system 1 asks "which categories belong to the post the most?" whereas system 2 asks "which posts belong to the category the most?", sort of. I think most of the time, categorization is useful for system 2's purpose (ie. searching for articles in a specific category), and when you want to know the categories of a specific post before reading it, system 2 should still be able to play system 1's role.

also, system 2 shows the average instead of the sum, which might better represent the categorization. on the other hand, knowing the number of votes that the categorizations received is useful to know how large the sample was.

both systems seems to allow for 5 options, so are equally fine-grained (although maybe system 2 for more as it seems you can select two options from the video?)

voting system is more intuitive (flows better?) because we already use it

I wonder if the post writer should have more votes or something

Voting seems to fit if you're okay with things not fitting together. If you want things to fit together, a set of "Economic" sequences/posts with an intentional order might work better.