Tagging Open Call / Discussion Thread

post by Ruby · 2020-07-28T21:58:25.087Z · score: 56 (16 votes) · LW · GW · 85 comments

Contents

  Why is tagging valuable?
  How do I help tag?
    Option 1: Dive right in!
    Option 2: Some helpful hints
      Good places to start
    Growing a community
None
84 comments

You’ve listened  to the LessWrong team talk about our new tagging feature for months. First a steady drip of we’re working on it, then announcements of various milestones like you can now filter Coronavirus in or out and anyone can create tags. Well, now, it's an open call for taggers.

We’ve sufficiently validated the core idea and developed enough tech that we’re ready to turn to the community in helping us gain complete tag coverage of LessWrongs’ 10-year corpus. 

That means:

  1. ensuring all the important concepts have been captured in high-quality tags [? · GW]
  2. all posts have been tagged with relevant tags
The new Concepts [? · GW] page

Why is tagging valuable?

Skip this section if you just want to know the how!

Multiple reasons, but I'm going to focus on one that is very dear to me.

One of the major goals of LessWrong [LW · GW] is intellectual progress on important problems. As far as I have seen, all major human breakthroughs built upon other breakthroughs. Later thinkers built upon earlier ones, or better yet, great thinkers built upon each others ideas. It's a common story, but one example from my quest to answer Why wasn't science invented in China? [LW · GW]: Francis Bacon didn't invent the modern scientific method from nowhere. Aristotle, Grosseteste, Roger Bacon were all part of the tradition before him.

I like to frame this cumulative way that progress is made as a “sustained conversation” that thinkers maintain over time. Over decades or centuries, some thinkers focus on the same ideas and pass knowledge between them, thereby pushing the frontier of what's collectively known so that more progress can be made.

However, this requires a medium of conversation. There has to exist some way for the thinkers to find each other and say things to each other. And for new people to catch up and join in on the conversation.

It's easy to have a brief conversation in a given time or place. It's much harder to sustain a conversation around the globe and over years. It would seem that great progress can suddenly happen if a new medium of conversation is provided. For example, the meetings and journal of the Royal Society allowed top scientists of Europe to converse throughout the 17th and 18th centuries to great effect. In the 150 years after the founding of the Royal Society, more than half of the scientists who made major scientific discoveries in that period were members. Causality is hard to prove in those case, but it seems linked.

You can see where this is going. Tagging is a way to sustain conversations over time. Right now, it's easy to have conversations on LessWrong about posts and topics being discussed this week. If a post is on the frontpage, 1) you're much more likely to find it and therefore be able to build upon it, and 2) if you comment on it, people are likely to see your comments and reply.

Suppose, however, that you're interested in anthropics. There hasn't been a LessWrong post on anthropics in the last four months, yet, over 11 years LessWrong had 81 posts on that topic [? · GW], some of them which are pretty darn good!

The point of tagging is that people can contribute knowledge to LessWrong's corpus, and have interested others find their contributions weeks, months, or years later. We want that when people contribute to LessWrong, they know they're contributing to something lasting. This isn't a news or entertainment site where posts are just part of a weekly cycle, they get some limelight, then are forgotten to the world. No. We're trying to build a goddamn edifice here. 

Let's sustain some conversations.

How do I help tag?

Option 1: Dive right in!

Though we have some guidelines, it's totally great to just go to post pages and start tagging them with what feels like the right tags. You can even create yet-to-exists tags without worrying too much. Better you dive in and we do some clean-up than you don't get started because it's too much work to get started.

Option 2: Some helpful hints

We've worked to prepare answers to all the questions we've encountered so far in the Tagging FAQ [LW · GW]. It covers and when and when not to tag, guidelines for creating tags, and some notes on tag voting. Ultimately, we'll aim to fix up all tags to be in-line with the style guide described there.

 Feel free to comment there with any questions not yet covered.

Good places to start

It's a good idea to start by becoming familiar with LessWrong's existing tags. You can see them on the new Concepts [? · GW] page. Then are a couple of tagging strategies:

Tag-First

Post-First

You might find that you end up iterating between the two approaches.

Growing a community

We'd like to build a small community around taggers – the people who maintain the ontology of LessWrong's library ensuring that desired information can always be found. 

Soon we'll have Discussions Pages for every tag, but in the meantime, if you want to connect with others about tagging, start a comment thread on the Tags Discussion/Talk Page [LW · GW].

If you have any questions whatsoever, please comment here, DM me (or the rest of the team), or email us at ruby@lesswrong.com or team@lesswrong.com

Thanks!

85 comments

Comments sorted by top scores.

comment by Raemon · 2020-08-03T22:24:08.268Z · score: 5 (3 votes) · LW(p) · GW(p)

Note: I currently lean towards changing the Progress Bar metric from "tagged posts over 25 karma" to 35 or 40 karma. 

The original reason we went with 25 karma was an awkward compromise due to the LW2.0 karma inflation – old upvotes were only worth 1 point, now regular upvotes are worth 2 for most longtime users, and strong upvotes mean the average is more like 3-4. We haven't gotten around to re-running the old vote history with the new vote-weighting, and that means that old (often great) posts have much lower karma than modern posts.

We plan to bring the old and new votes in sync someday, but didn't have time to do it this week. 

For modern posts, the threshold I'd have preferred to set was ~50 karma. This was roughly the equivalent of 25 back-in-the-day (hence the original metric). But I don't really want to make people feel obligated to tag a bunch of mediocre modern posts – I'd rather taggers start shifting their efforts towards improving tag descriptions (turn stubs into full fledged A or B tier tags [? · GW]), and thinking about how the tag ontology fits together (i.e. are some tags duplicates? which tags are related?)

My current guess is we should set the threshold to 40, and then I'm just going to strong upvote a bunch of older posts that deserve it to bump them over the threshold. 

(Meanwhile, to all the users who have doing doing tons of tagging: thanks!)

comment by Ruby · 2020-07-31T02:18:38.935Z · score: 10 (5 votes) · LW(p) · GW(p)

Someone mentioned that they thought the Concepts / Tag Portal was really nifty and they only just got round to looking at it, and that they thought it was motivating for tagging. I probably should have included a screenshot in the text (just added), but here's a comment with a larger one:

www.lesswrong.com/tags/all [? · GW]

The new Concepts [? · GW] page
comment by Multicore (KaynanK) · 2020-07-29T18:47:25.231Z · score: 31 (11 votes) · LW(p) · GW(p)

A way I can contribute to the site without having to come up with brilliant original ideas? Excellent!

comment by DanielFilan · 2020-08-03T01:12:47.925Z · score: 4 (2 votes) · LW(p) · GW(p)

This is probably a joke, but in my experience, explaining other people's ideas is also a valued contribution if you explain it well and people are interested in the ideas.

comment by Ruby · 2020-08-03T01:50:14.012Z · score: 9 (5 votes) · LW(p) · GW(p)

I'm pleased to say that based on the bunch of tagging Multicore has been doing (we'll make a tagging activity dashboard generally available soon), the above comment was entirely genuine.

comment by Multicore (KaynanK) · 2020-08-04T02:25:30.866Z · score: 29 (8 votes) · LW(p) · GW(p)

PSA: Voting on relevance is an important, underserved, and easy to contribute to area of the tagging system.

One person can create a tag, make a good description, and find a bunch of posts that fit it, but it takes multiple people's votes to create a decent ordering of posts from most to least relevant. Which posts are listed first will be an important part of the user experience.

This will be especially important for the more crowded tags, like the core tags, history, math, science, statistics, ai risk, and so forth.

Contributing can be as easy as just going through the list and upvoting posts that you've read and think are a good fit for the tag.

Edit: It would be nice to have a spreadsheet sorting tags by something like average relevance karma per post, to identify which tags most need votes.

comment by Raemon · 2020-08-04T19:52:14.408Z · score: 4 (2 votes) · LW(p) · GW(p)

Hey Multicore, I'd be interested in your thoughts on this alternate voting system [LW · GW] I had proposed awhile ago, which (among other goals) aims to shift things such that it's less necessary for multiple people to collaborate to vote on stuff.

The team has gone back and forth on whether it'd be an improvement, and/or whether it's enough of an improvement to be worth the dev-cost to switch.

comment by Multicore (KaynanK) · 2020-08-04T20:54:56.837Z · score: 7 (4 votes) · LW(p) · GW(p)

I agree with most of your analysis in the comments (many downsides to karma, multiple choice has some advantages intuition-wise and makes it easier for a single user to make an ordering), but I thought of a couple more points. My mind seems to only be coming up with downsides of the multiple choice system, which might be because I'm prone to rationalizing why the status quo is good.

  • Multiple choice has strategic voting implications too. If I think a 150 karma post and a 50 karma post are both "Top" relevance, but that the 50 karma post is better, I might rate the 150 karma post as "high" or lower.
  • Multiple choice makes it harder to see where in the ordering your vote would make a post end up. Additionally, your vote either has no immediate effect or moves the post around by a lot, so a fine-grained adjustment is impossible. That might not necessarily be bad though, if post karma mattering is desired.
  • If ordering is based only on the median vote, this makes it easy for a troll to vandalize a tag page even when the tagging system is mature. Just put the tag on a bunch of posts that don't already have it and rate them all "Top". With karma, the post order is more stable once a lot of people have voted. (This is the double edge of making it easy for a single person to have a big impact.)

However, these concerns balance out against the benefits you listed, so overall I don't have a strong opinion on which is better.

comment by Yoav Ravid · 2020-07-31T12:00:58.229Z · score: 15 (6 votes) · LW(p) · GW(p)

Making tags is fun!

Here are some tags i made:

Tagging [? · GW]

Open Problems [? · GW]

Summaries [? · GW]

Prepping [? · GW]

Meetups (topic) [? · GW]

Category Theory [? · GW](this one needs work because i know very little about category theory, but it seemed like a tag that should exist)

Urban Planning / Design [? · GW]

comment by adamShimi · 2020-07-31T14:55:34.541Z · score: 4 (2 votes) · LW(p) · GW(p)

Hey! I think it's cool that you created new tags. That being said, I do think that your description of category theory is not only a stub, but completely uninformative and dismissive about a mathematical field that has almost 70 years of work in it. I do think that explaining the controversy on the applicability of category theory is valuable, as we should question whether to use it for rationality and AI. But that should be a note at the end of the tag description, not the entire content of it.

(Note that I didn't change the tag description, because I don't want to force a change if I'm the only one thinking that. Maybe the point is only to describe how the word in the tag is used in LW, in which case the current tag description might work.)

comment by Yoav Ravid · 2020-07-31T16:34:07.839Z · score: 8 (2 votes) · LW(p) · GW(p)

I agree with you, i struggled writing the description for it as i know very little about it, i just saw that it doesn't exist yet and went ahead creating it (maybe it would have been better if i left it with no description?). so i say go ahead and edit it, you'll surely do a better job than me :)

comment by adamShimi · 2020-08-01T12:07:58.572Z · score: 12 (3 votes) · LW(p) · GW(p)

I wrote a first version of the new tag description, I might rewrite some parts of it later. ;)

comment by Yoav Ravid · 2020-08-01T12:13:17.321Z · score: 1 (1 votes) · LW(p) · GW(p)

Thanks, much better :D

comment by Ruby · 2020-07-31T19:21:14.423Z · score: 7 (4 votes) · LW(p) · GW(p)

+1 to be people creating tags they think are needed even if they're not sure about the description.

comment by Yoav Ravid · 2020-07-31T19:58:22.147Z · score: 1 (1 votes) · LW(p) · GW(p)

Thanks :)

In that case is it better to not write a description or write something knowing there's a high chance it'll be wrong (and leaving a note about it)?

comment by Ruby · 2020-07-31T22:14:06.135Z · score: 4 (2 votes) · LW(p) · GW(p)

My general philosophy is "better a description than no description". 

comment by Ben Pace (Benito) · 2020-07-31T20:19:28.346Z · score: 4 (2 votes) · LW(p) · GW(p)

I think it was the right call to write the tag descriptions that you did :) 

Yeah, I'd continue to write something knowing there's a high change it'll be wrong.

comment by Ben Pace (Benito) · 2020-07-31T17:37:56.092Z · score: 5 (3 votes) · LW(p) · GW(p)

ahahaha that tag description is hilarious (no offesne Yoav, thanks for making the tag!). adamShimi, please feel very welcome to change that tag description.

comment by Yoav Ravid · 2020-07-31T17:40:58.732Z · score: 5 (3 votes) · LW(p) · GW(p)

Well at least i made someone laugh :D

comment by Ruby · 2020-07-31T19:28:36.173Z · score: 3 (2 votes) · LW(p) · GW(p)

As I've said a bit elsethread, I'm very in favor of 1) people creating tags they think should exist even if they don't know the topic well, 2) other people jumping in improving them where they see possible. I hope talk pages will make it easy to discuss any changes made, but most of the time I expect they'll just be all-round welcome improvements that don't need debating.

In short, if you think there's an improvement to be made, go for it! Even if you're not sure that anyone else agrees. Let them object after the fact.

More philosophical, I think we generally want to define terms like category theory in their proper standard usage. If the LW usage is off, we probably want to correct that or at least note the divergence in any description. (I don't remember if the S1/S2 description, but it would be good if it noted the difference between academic meaning and how it gets used around here).

If you end up writing an improved Category Theory description, I look forward to reading it.

comment by Ruby · 2020-07-31T19:22:20.856Z · score: 2 (1 votes) · LW(p) · GW(p)

Thanks so much for diving in and doing so much! I'm so glad it's fun. :)

comment by Bucky · 2020-08-04T16:18:38.324Z · score: 11 (4 votes) · LW(p) · GW(p)

Are there any thoughts on external links for tag wiki pages? I was looking at the social status tag, for example, and there are a few overcomingbias / ribbonfarm posts which I think would be useful but whether / how best to incorporate them isn't clear to me

comment by ChristianKl · 2020-08-04T19:50:57.362Z · score: 5 (3 votes) · LW(p) · GW(p)

How about creating link posts for them?

comment by Raemon · 2020-08-04T20:39:08.518Z · score: 2 (1 votes) · LW(p) · GW(p)

I think this is pretty reasonable. (Ideally the link posts also come with some summaries. In some cases it makes sense to ping the author about fully crossposting it)

comment by Ruby · 2020-08-04T16:57:51.531Z · score: 5 (3 votes) · LW(p) · GW(p)

My thought is they're great! I've generally hoped the descriptions-texts would contain more of them. I explicitly listed them as a factor in meriting a  A-Class or B-Class tag-grade [? · GW].

I think a good way to do this is to have an "External Resources" section as part of the tag description that lists the external resources (with or without some context on them).

comment by Yoav Ravid · 2020-08-04T17:49:46.314Z · score: 3 (2 votes) · LW(p) · GW(p)

I wondered about that too since, for example, open problems in group rationality isn't on LessWrong, but is probably the main article on Group Rationality [? · GW].

comment by Ruby · 2020-08-04T21:49:24.865Z · score: 2 (1 votes) · LW(p) · GW(p)

I vaguely remembered there was something like that and was surprised it wasn't on the tag, so seems pretty good if it's there in some form.

comment by Gyrodiot · 2020-07-28T22:06:49.797Z · score: 10 (3 votes) · LW(p) · GW(p)

Thanks for the project, and the FAQ. I shall contribute.

Is there a way to retrieve the old tags from LW 1.0? I remember they were used to index Open Threads, for instance. I can't remember the details but that could be a good way to jumpstart some tags.

comment by Ruby · 2020-07-28T22:20:19.026Z · score: 16 (6 votes) · LW(p) · GW(p)

Awesome! Thanks! Don't hesitate hesitate to ask for any help making the process easier.

Re: LW1.0 tags -- good question. I believe they they were implemented via the old LW1.0 wiki and you can still view all those pages and their tagged posts. https://wiki.lesswrong.com/wiki/Special:AllPages

We've been planning to probably import that content, we estimate about 100 high quality articles/tags, but it's been waiting behind other dev work that seemed higher priority, e.g., Talk Pages for tags. 

We've also got our eyes on the Arbital content, however that's a lot more work since we don't currently have all the features that Arbital uses.

comment by Yoav Ravid · 2020-08-03T13:35:41.266Z · score: 7 (3 votes) · LW(p) · GW(p)

There now seems to be over a 100 Sequences [? · GW](depending on how one counts). I think it would be good to have tags for Sequences, Especially since they're not easily searchable through a search engine (or not that I'm aware of, at least).

comment by adamzerner · 2020-08-01T19:46:04.436Z · score: 7 (4 votes) · LW(p) · GW(p)

When I click "Add Tag", this is what I see:

Non-expanded view of Add Tag

Then I clicked to show more, because I know there are a lot more tags and want to make sure that if I tag a post it has all of the proper tags (because if I don't it'll be marked as tagged and it's likely that no one will return to it to add the proper tags):

Expanded view of Add Tag

But this view isn't organized well like the concepts portal is (below), so I felt the need to skim through each individual tag, which took a long time. Seems like it'd be a good idea to organize the above view to look more like the below view.

Expanded view of Add Tag

comment by Raemon · 2020-08-01T19:56:34.144Z · score: 3 (2 votes) · LW(p) · GW(p)

Agreed. It's just actually a bit annoying to get it to work right, due to how the algolia search function works. 

comment by Ruby · 2020-08-01T20:13:24.221Z · score: 3 (2 votes) · LW(p) · GW(p)

I think that adds to the reasons to perhaps replace Algolia.

comment by Raemon · 2020-08-01T20:42:56.138Z · score: 5 (3 votes) · LW(p) · GW(p)

I don't think this issue would go away with replacing algolia. The problem is a sort of generic "algolia is faster than making a database query, but that speed comes with some default settings that require re-wiring, which I'm sure is possible but requires some upfront costs that aren't part of my usual workflow. I think any search engine would come with the same issue."

comment by Ruby · 2020-08-01T20:52:30.096Z · score: 3 (2 votes) · LW(p) · GW(p)

(Don't know how much the rest of LW wants to hear our internal dev discussions, but) it's also things like Algolia, at least on our current plan, doesn't have personalization, e.g. to recommend tags a person previously used or tags we algorithmically guess would apply this post but would want a human you check. 

Mostly going off Oli saying leaving Algolia is would be the way forward here. You might be right that no other solution will be better for this particular thing.

comment by habryka (habryka4) · 2020-08-01T21:25:47.792Z · score: 3 (2 votes) · LW(p) · GW(p)

I am pretty confused what any of this has to do with Algolia. The primary problem to me appears to be that we don't actually have a large fraction of the tags categorized in the tag hierarchy displayed on the All Tags page. We could show you a copy of the tag page table, but that would omit a lot of new tags, and also probably not be dense enough. We could develop some custom UI for that menu to group them, but that's mostly a bunch of work (and doesn't have super much to do with Algolia). 

The site search will probably always have somewhat different constraints than normal database operations (in particular if we want to stay within the autocomplete paradigm), so I don't think anything about this would get easier if we switch away from Algolia (things like this are actually a domain where Algolia is pretty great). 

comment by Ruby · 2020-08-01T22:28:14.092Z · score: 5 (3 votes) · LW(p) · GW(p)

I stand corrected and I hope Algolia is accepts my apologies for the slight. The actual table I don't think is much a possibility, if desirable at all, but structured things are good. The alternative is just ordered things, if we can accurately predict which things are likely.

comment by habryka (habryka4) · 2020-08-02T00:21:02.803Z · score: 2 (1 votes) · LW(p) · GW(p)

Yeah, ok. I do think personalization is blocked on Algolia, and I didn't really think about this as a potential solution to this (but it totally is). So yeah, maybe slighting Algolia was the right call.

comment by Yoav Ravid · 2020-07-31T06:21:41.763Z · score: 5 (3 votes) · LW(p) · GW(p)

The progress bar in the main page is super cool. I'm curious, how is it being calculated?

Edit: oh, when you hover over it it says "X out of 4735 posts have been tagged (filtered for 25+ karma)
So i guess that's how

comment by Yoav Ravid · 2020-07-31T16:43:26.542Z · score: 5 (3 votes) · LW(p) · GW(p)

Is it just me or is the number of posts tagged that's shown on hover is going down with time?

comment by habryka (habryka4) · 2020-07-31T17:56:34.751Z · score: 15 (5 votes) · LW(p) · GW(p)

Ahh, hmm. That is embarrassing. Hmm, I wish I had a better excuse for this. Hmm... 

I mean, look over there a three-headed monkey!

(Will be fixed within the hour)

Edit: And it's fixed. Sorry about that!

comment by adamShimi · 2020-08-02T18:33:05.315Z · score: 4 (3 votes) · LW(p) · GW(p)

Just commenting to say it's pretty cool to see the bar filling up and the number of tagged posts growing up. Thanks to all the taggers!

comment by Ruby · 2020-08-02T21:29:52.293Z · score: 2 (1 votes) · LW(p) · GW(p)

Woop!

We haven't yet built any ways to recognize or reward the taggers, and I'd really like to. Any suggestions for how to do that?

Also, I don't want to shower a lot of attention on someone if they don't want it. We did say that tagging activity would be public, but I don't think that's salient yet. If you've tagged a lot and don't want a shout-out, please DM me before we get around to doing something proper.

One short-term idea: if you've been tagging a lot, you can comment here and I can tell you how many tags you've applied to date. Others could then upvote your comment if they wanted to, to say thanks. Also general intangible prestige.

comment by DanielFilan · 2020-08-03T01:35:38.107Z · score: 13 (6 votes) · LW(p) · GW(p)

We haven't yet built any ways to recognize or reward the taggers, and I'd really like to. Any suggestions for how to do that?

Publish a book of the best instances of people applying tags to posts in 2020.

comment by adamShimi · 2020-08-03T18:37:15.685Z · score: 8 (2 votes) · LW(p) · GW(p)

Karma is nice. Maybe simply an appreciation post at some point, which could still not name people. Just let them know that they are appreciated.

I don't know if that's possible, but another option might be some sort of "rank" or "badge" for top taggers. That being said, one might ask why have ranks only for this specific case, and not in general.

comment by Raemon · 2020-08-02T18:15:41.414Z · score: 4 (2 votes) · LW(p) · GW(p)

Tagging meta question: 

There's a Calibration (Probability) tag. How important is it to keep that distinct from other forms of calibration (i.e. if you think some parameter will be within particular bounds, those bounds tend to be correct). 

Prompted by this post on time calibration

I suppose that all calibration is implicit probability calibration (i.e. if I think something will take between 15 and 45 minutes, I'm sort of implicitly claiming it has a high probability of being so, even if I didn't concretely decide it was my 90% confidence interval). But, if all calibration can be reformulated as probability, do we need the "(probability)" disambiguation?

comment by Ruby · 2020-08-02T18:58:33.133Z · score: 4 (2 votes) · LW(p) · GW(p)

Not all calibration is probability calibration, e.g., calibrating my scales or voltmeter, but as you suggest, calibration discussion on LessWrong is effectively calibration about credences/probabilities. Not worth keeping finer gradations distinct.

But I think the disambiguation is good because it explains the tag to someone new on LessWrong and doesn't rely on your already  knowing the content, so I'm in favor of the disambiguation. The way we use calibration is a our own jargon, so good to explain a bit what we mean just in the title.

comment by Raemon · 2020-08-02T19:39:33.136Z · score: 2 (1 votes) · LW(p) · GW(p)

I can see the case for that, but FYI it just made me go make this meta comment rather than intuitively classifying a calibration post. I think it might be fine to have the disambiguation live in the text rather than the title.

comment by Ruby · 2020-08-02T21:10:49.300Z · score: 3 (2 votes) · LW(p) · GW(p)

Epistemic status: I'm generally pro disambiguations in parentheses, I was the one who advocated we borrow the practice from Wikipedia. I'm really not sure in the case between Calibration and Calibration (Probability), so I'm just trying to think through this with others. 

The tag description is this:

Do the events that you give a 70% probability in advance, actually end up happening 70% of the time? 

It's pretty brief, I'm guessing that didn't clarify enough. I actually suspect here that if the description had been like the following, clarification wouldn't have been necessary. Raemon?

Someone is probability or credence calibrated if the things they predict with 70% chance of happening in fact occur 70% of the time. Importantly, calibration is not the same as accuracy. Calibration is about accurately assessing how good your predictions are, not making good predictions. Person A, whose predictions are marginally better than chance, e.g. 60% of them come true, and who knows that, is well-calibrated. In contrast, Person B, whose predictions are 90% accurate, yet thinks they are 99% accurate, is more accurate than Person A while being less well calibrated.

Knowing how good your predictions are is a key rationalist skill. Among other things, being calibrated lets you make good bets [Link to Betting tag]/make good decisions [link Planning & Decision-making tag], communicate information helpfully to others if they know you to be well-calibrated [link to Group Rationality], and helps prioritize which information is worth acquring [link to VoI tag].

Note that calibration applies to all expressions of quantified confidence in beliefs/predictions [reference: Anticipate Experiences]. For example, calibration applies to whether a person's 95% confidence intervals capture things 95% of the time. Or if their 80% chance of completion estimates are met 80% of the time (not much more or less). Trivially, odds ratio placed on things are convertible to probabilities.

See also: prediction & forecasting

I think this makes it much clearer that "tends to be correct" is always a quantitive/probabilistic statement beneath the hood.

Hmm, what this makes me think is really it's about calibrating credences, which isn't standard jargon anyway. So maybe just plain "Calibration" is better than "Probability". Or maybe Calibration (belief strength)?

My main thought now is the issue wasn't the name so much as lack of good explanation for the topic. [No complaint against the tag creator – it was a good tag to make.] I'd kind of like to have a big tag description writing push sometime after the plain "tag at all" push since so far few tags have good explanations. But we'd have to decide we're definitely moving more in this wiki-ish direction.

Yeah, I'd propose Calibration or Calibration (belief-strength).

I dislike Probability Calibration because I dislike leading adjectives/modifiers and prefer the main thing to be the first word in the noun phrase (some languages like Hebrew do this). I expect people to be looking for the core thing, e.g. Relationships, "R", and if you put modifiers in front, e.g. "Business", "Personal/Interpersonal", "Romantic", "Conceptual", you then require someone to guess which modifier you used, and also split up Relationship tags from being adjacent in an alphabetical list.

I think having as much in the title as possible is better than in the text, just because even triggering the hover-over is 10-20x costly than just skimming all the titles, and also doesn't work on mobile where are no hover-overs. I think if someone's looking over the tags list and sees "Calibration (belief-strength)" they have much more of an idea of what tag is about than just Calibration which is pretty opaque to an outsider. 

comment by Kaj_Sotala · 2020-08-02T21:55:29.873Z · score: 4 (2 votes) · LW(p) · GW(p)

I dislike Probability Calibration because I dislike leading adjectives/modifiers and prefer the main thing to be the first word in the noun phrase (some languages like Hebrew do this). I expect people to be looking for the core thing, e.g. Relationships, "R", and if you put modifiers in front, e.g. "Business", "Personal/Interpersonal", "Romantic", "Conceptual", you then require someone to guess which modifier you used, and also split up Relationship tags from being adjacent in an alphabetical list.

There is that, but at the same time, you probably wouldn't want tags like Experiences (Anticipated) or Induction (Solomonoff). I don't have any principled argument for this, but to me "Probability Calibration" feels more like one of those examples. It being put alphabetically close to "Probability" may also be good.

(I also keep feeling confused by Relationships (Interpersonal) each time I see it, though that's probably in part because there's no other 'Relationships', so I just think 'well what other relationships could you even mean' and then don't find another Relationships that it would be contrasted with.)

comment by Ruby · 2020-08-02T22:25:14.703Z · score: 4 (2 votes) · LW(p) · GW(p)

That's a fair point to raise. I think it's more work to give an explicit theory for why those are different. Something like those are both technical terms/jargon and they don't belong to a broader class of things that we might be discussing. In a world where there were three types of induction we discussed, might then go for Induction (Solomonoff). Also that they're the actual phrase people use. I don't think I can remember anyone saying "probability calibration" ever.

With Relationships (Interpersonal), I think it makes sense because to me, the default way to read "relationships" is specifically romantic relationships. Like if your friend says "I'm reading a book about relationships", what do you assume? To make it clear the tag also covers friendship, family, and work relationships I think actually does require some disambiguation even if the site doesn't have any other relationship tags right now.

comment by Yoav Ravid · 2020-08-03T05:27:00.042Z · score: 1 (1 votes) · LW(p) · GW(p)

So perhaps we should also have Relationships (Romance)? :)

comment by Ruby · 2020-08-03T07:19:21.188Z · score: 2 (1 votes) · LW(p) · GW(p)

Indeed. <3

comment by Yoav Ravid · 2020-08-03T06:16:12.342Z · score: 1 (1 votes) · LW(p) · GW(p)

I went ahead and edited the tag to have your description

comment by Ruby · 2020-08-03T07:24:43.185Z · score: 2 (1 votes) · LW(p) · GW(p)

Many thanks!!

comment by Kaj_Sotala · 2020-08-02T20:25:04.304Z · score: 2 (1 votes) · LW(p) · GW(p)

"Probability Calibration" rather than "Calibration (Probability)" feels like a more natural name for the tag, while keeping the disambiguation.

comment by Ruby · 2020-08-02T21:11:56.362Z · score: 2 (1 votes) · LW(p) · GW(p)

See my comment in the other thread about my argument for modifiers after the main word. It's not LW team consensus, just something I've been pushing for. 

comment by Yoav Ravid · 2020-08-01T10:21:54.799Z · score: 4 (3 votes) · LW(p) · GW(p)

Would be nice to be able to see tags by date created, or something to know what new tags are created, so we can take a look and think if there's anything that needs to go there.

comment by Ruby · 2020-08-01T17:33:04.682Z · score: 3 (2 votes) · LW(p) · GW(p)

That makes sense! That's the kind of thing we should build into a proper tagging dashboard.

This spreadsheet has tags sorted by last changed (edited or posts added) but I'll need to add in an additional column in order to sort by creation date. I can get around to it soon, if it's helpful.

comment by Gunnar_Zarncke · 2020-07-31T00:07:28.880Z · score: 4 (2 votes) · LW(p) · GW(p)

I have tagged a few posts from the top of the spreadsheet but not too many because it caused me reading too many old posts...

I have added the tag Habits; hope that makes sense. I'm not too clear about the taxonomy.

How frequently is the spreadsheet updated? UPDATE: Every 5 Min according to the OP.

comment by Ruby · 2020-07-31T02:16:24.409Z · score: 5 (3 votes) · LW(p) · GW(p)

but not too many because it caused me reading too many old posts...

Darn! A pernicious failure mode indeed.

comment by Raemon · 2020-07-31T00:26:38.600Z · score: 4 (2 votes) · LW(p) · GW(p)

Thanks! Yesterday I was looking at the Martial Art of Rationality and trying to figure out what to tag it, and kinda gave up. I believe you created the tag for that? (I think it was the first tagged post). Kudos, it was a good solution I think.

comment by ESRogs · 2020-07-29T19:18:36.024Z · score: 4 (2 votes) · LW(p) · GW(p)

We'd like to build a small community around taggers – the people who maintain the ontology of LessWrong's library ensuring that desired information can always be found.

Maybe this is a dumb question but, is this actually needed?

Can we get what we want with people just randomly adding tags when they notice? Do we need to have people specializing on this?

I'd expect that a bunch of work would be needed up front to get the tag system into a good state, but I'd think most of that work has been done already (by the LW team, and others). And then going forward I'd expect much less work to be required. Am I missing something?

comment by habryka (habryka4) · 2020-07-29T19:34:53.997Z · score: 7 (4 votes) · LW(p) · GW(p)

I think tagging is actually pretty hard. Like, by default you get a ton of synonyms of the same concepts, and there aren't good redirects, and the tags don't have good descriptions, and there is lots of ambiguity, and when someone creates a new tag old posts don't reliably get tagged. Our tagging system is also more similar to being a wiki, and in-general my research into wikis suggests that basically all functional ones are maintained by a relatively small group of highly dedicated editors, and that it generally doesn't work to just have everyone randomly edit and add things.

comment by ESRogs · 2020-07-29T21:55:35.077Z · score: 4 (2 votes) · LW(p) · GW(p)

That's helpful context. Makes sense, thanks!

comment by Kaj_Sotala · 2020-07-29T13:24:11.855Z · score: 4 (2 votes) · LW(p) · GW(p)
Alternatively, we have an automatically updating spreadsheet (every 5 min) that tracks the tags on the most viewed posts according to our data and their current tags.

Note that this spreadsheet seems to open by default to the "sorted by karma" tab, and you have to manually switch to the "sorted by view rank" tab (spent half a minute wondering whether the link was incorrect before happening to look at the tab list).

(Also: Oh wow, our most viewed post [? · GW] in one with four karma?)

comment by habryka (habryka4) · 2020-07-29T17:10:03.375Z · score: 8 (4 votes) · LW(p) · GW(p)

Note: This is most viewed for the last 30 days I think. That post in particular tends to show up at the top every few months whenever you get closer to U.S. College application periods. 

And yeah, sometimes the posts that get a ton of views really aren’t very good. For a while one of our most viewed post was one at -4 karma called “The Effects of Religion [Draft]”. It‘s been one of the reasons why I’ve been hesitant to include views as an easily accessible metric on the site, because I know how frequently it diverges from quality.

comment by Ruby · 2020-07-29T18:19:59.397Z · score: 2 (1 votes) · LW(p) · GW(p)

*Last 90 days

comment by Yoav Ravid · 2020-08-03T13:32:08.325Z · score: 2 (2 votes) · LW(p) · GW(p)

I wanted to create a 'Law and Legal Systems' tag, but then i saw there's a Government [? · GW] tag, currently only with 2 posts. should these tags be separate, or should i change the current tag to "Law, Legal Systems And Government"?

comment by Ruby · 2020-08-03T15:36:46.097Z · score: 2 (1 votes) · LW(p) · GW(p)

They seem like different things to me.

comment by Multicore (KaynanK) · 2020-07-31T19:52:31.861Z · score: 2 (2 votes) · LW(p) · GW(p)

The tags page is occasionally suddenly replacing itself with the message "Error: TypeError: Cannot read property '_id' of null", forcing me to reload the page. Has anyone else seen this?

Edit: I also got the same error on the page for a post, when I added a tag, the server response was slow, and I tried to add it again.

comment by Ruby · 2020-08-01T01:47:08.260Z · score: 2 (1 votes) · LW(p) · GW(p)

Thanks! W

e're looking into it.

comment by Yoav Ravid · 2020-07-31T20:24:45.922Z · score: 1 (1 votes) · LW(p) · GW(p)

Happened to me too (though perhaps a slightly different error message, not sure)

comment by ESRogs · 2020-07-29T19:05:27.668Z · score: 2 (1 votes) · LW(p) · GW(p)

It's a good to start by becoming familiar with LessWrong's existing tags. You can see them on the new Concepts page.

In the text, Concepts shows up green, like it's supposed to be a link, but it doesn't go anywhere. Was it supposed to be a link?

comment by ESRogs · 2020-07-29T19:06:37.100Z · score: 2 (1 votes) · LW(p) · GW(p)

Also, maybe it was supposed to be "It's a good idea to start..."?

comment by Ruby · 2020-07-29T19:10:46.161Z · score: 5 (3 votes) · LW(p) · GW(p)

Thanks, fixed on both accounts!

comment by ESRogs · 2020-07-29T21:54:08.030Z · score: 5 (3 votes) · LW(p) · GW(p)

The Concepts link is still not going anywhere for me :-/. When I inspect on Chrome, it shows up like this:

comment by habryka (habryka4) · 2020-07-29T22:28:40.033Z · score: 6 (3 votes) · LW(p) · GW(p)

Now actually fixed (there was a typo in the URL).

comment by Multicore (KaynanK) · 2020-08-02T18:19:45.758Z · score: 1 (1 votes) · LW(p) · GW(p)

Diseased disciplines: the strange case of the inverted chart [LW · GW] is an interesting case because tagging it correctly feels like a spoiler.

comment by Yoav Ravid · 2020-08-02T20:14:23.733Z · score: 1 (1 votes) · LW(p) · GW(p)

After i read your comment i decided to read the post. i saw the tag programing*, but forgot about it midway and it didn't spoil the surprise for me - still i think you're probably right it would for other people.

Anyway, i also added it to information cascades, as it seems relevant and doesn't spoil it.

*not sure how to do a spoiler tag

comment by Yoav Ravid · 2020-07-31T09:54:05.096Z · score: 1 (1 votes) · LW(p) · GW(p)

Would be nice to be able to add tags from the drop-down menu to the right of posts (in places like the homepage and users profiles). This would speed up the process by a lot.

comment by Ruby · 2020-08-01T01:46:44.424Z · score: 2 (1 votes) · LW(p) · GW(p)

I agree. Currently, there's the "Edit Tags" button that gets you there, but we've been meaning to replace that with the "AddTag" button directly. Thanks for the feedback.

comment by Raemon · 2020-08-01T01:53:20.979Z · score: 2 (1 votes) · LW(p) · GW(p)

(quick note: the Edit Tags button is actually admin only. We can make that available to everyone soon though)

comment by Ruby · 2020-08-01T02:00:54.178Z · score: 2 (1 votes) · LW(p) · GW(p)

Ohh. I'm terribly sorry then, everyone. We'll get that fixed.