Tagging Open Call / Discussion Thread
post by Ruby · 2020-07-28T21:58:25.087Z · LW · GW · 118 commentsContents
Why is tagging valuable? How do I help tag? Option 1: Dive right in! Option 2: Some helpful hints Good places to start Growing a community None 118 comments
You’ve listened to the LessWrong team talk about our new tagging feature for months. First a steady drip of we’re working on it, then announcements of various milestones like you can now filter Coronavirus in or out and anyone can create tags. Well, now, it's an open call for taggers.
We’ve sufficiently validated the core idea and developed enough tech that we’re ready to turn to the community in helping us gain complete tag coverage of LessWrongs’ 10-year corpus.
That means:
- ensuring all the important concepts have been captured in high-quality tags [? · GW]
- all posts have been tagged with relevant tags
Why is tagging valuable?
Skip this section if you just want to know the how!
Multiple reasons, but I'm going to focus on one that is very dear to me.
One of the major goals of LessWrong [LW · GW] is intellectual progress on important problems. As far as I have seen, all major human breakthroughs built upon other breakthroughs. Later thinkers built upon earlier ones, or better yet, great thinkers built upon each others ideas. It's a common story, but one example from my quest to answer Why wasn't science invented in China? [LW · GW]: Francis Bacon didn't invent the modern scientific method from nowhere. Aristotle, Grosseteste, and Roger Bacon were all part of the tradition before him.
I like to frame this cumulative way that progress is made as a “sustained conversation” that thinkers maintain over time. Over decades or centuries, some thinkers focus on the same ideas and pass knowledge between them, thereby pushing the frontier of what's collectively known so that more progress can be made.
However, this requires a medium of conversation. There has to exist some way for the thinkers to find each other and say things to each other. And for new people to catch up and join in on the conversation.
It's easy to have a brief conversation in a given time or place. It's much harder to sustain a conversation around the globe and over years. It would seem that great progress can suddenly happen if a new medium of conversation is provided. For example, the meetings and journal of the Royal Society allowed top scientists of Europe to converse throughout the 17th and 18th centuries to great effect. In the 150 years after the founding of the Royal Society, more than half of the scientists who made major scientific discoveries in that period were members. Causality is hard to prove in this case, but it seems linked.
You can see where this is going. Tagging is a way to sustain conversations over time. Right now, it's easy to have conversations on LessWrong about posts and topics being discussed this week. If a post is on the Frontpage, 1) you're much more likely to find it and therefore be able to build upon it, and 2) if you comment on it, people are likely to see your comments and reply.
Suppose, however, that you're interested in anthropics. There hasn't been a LessWrong post on anthropics in the last four months, yet, over 11 years LessWrong has accrued 81 posts on that topic [? · GW], some of them which are pretty darn good!
The point of tagging is that people can contribute knowledge to LessWrong's corpus, and have interested others find their contributions weeks, months, or years later. We want that when people contribute to LessWrong, they know they're contributing to something lasting. This isn't a news or entertainment site where posts are just part of a weekly cycle, they get some limelight, then are forgotten to the world. No. We're trying to build a goddamn edifice here.
Let's sustain some conversations.
How do I help tag?
Option 1: Dive right in!
Though we have some guidelines, it's totally great to just go to post pages and start tagging them with what feels like the right tags. You can even create yet-to-exists tags without worrying too much. Better you dive in and we do some clean-up than you don't get started because it's too much work to get started.
Option 2: Some helpful hints
We've worked to prepare answers to all the questions we've encountered so far in the Tagging FAQ [LW · GW]. It covers and when and when not to tag, guidelines for creating tags, and some notes on tag voting. Ultimately, we'll aim to fix up all tags to be in-line with the style guide described there.
Feel free to comment there with any questions not yet covered.
Good places to start
It's a good idea to start by becoming familiar with LessWrong's existing tags. You can see them on the new Concepts [? · GW] page. Then are a couple of tagging strategies:
Tag-First
- You might notice the absence of tags you think should exist. Great! You can create that tag now [? · GW] and then begin adding relevant posts to it.
- You might see a tag and be surprised at how few posts it has. Awesome, you can add any missing posts you know of to it from either the tag page or post page [LW · GW].
- This spreadsheet has all tags sorted by their Tag Grade [? · GW].
- If you're on a tag page, you can help ensure the most relevant posts are voted to the top [LW · GW].
Post-First
- Look at the list of high-karma posts here that don't yet have tags [? · GW]. See if they fit any existing ones, or whether we're missing a tag for a real cluster, then make it. This spreadsheet is a different lens on high karma posts. It displays tags currently applied and lets you ignore Core Tags [LW · GW] that are less informative.
- Alternatively, we have an automatically updating spreadsheet (every 5 min) that tracks the tags on the most viewed posts according to our data and their current tags. Causing those to have good tags is a high-leverage due to the high traffic.
- The sheet opens to a View of posts not having any non-core [LW · GW] tags, our current focus.
- If you're an author, check that all your own posts are appropriately tagged.
You might find that you end up iterating between the two approaches.
Growing a community
We'd like to build a small community around taggers – the people who maintain the ontology of LessWrong's library ensuring that desired information can always be found.
Soon we'll have Discussions Pages for every tag, but in the meantime, if you want to connect with others about tagging, please comment on this post.
If you have any questions whatsoever, please comment here, DM me (or the rest of the team), or email us at ruby@lesswrong.com or team@lesswrong.com
Thanks!
118 comments
Comments sorted by top scores.
comment by Raemon · 2020-08-03T22:24:08.268Z · LW(p) · GW(p)
Note: I currently lean towards changing the Progress Bar metric from "tagged posts over 25 karma" to 35 or 40 karma.
The original reason we went with 25 karma was an awkward compromise due to the LW2.0 karma inflation – old upvotes were only worth 1 point, now regular upvotes are worth 2 for most longtime users, and strong upvotes mean the average is more like 3-4. We haven't gotten around to re-running the old vote history with the new vote-weighting, and that means that old (often great) posts have much lower karma than modern posts.
We plan to bring the old and new votes in sync someday, but didn't have time to do it this week.
For modern posts, the threshold I'd have preferred to set was ~50 karma. This was roughly the equivalent of 25 back-in-the-day (hence the original metric). But I don't really want to make people feel obligated to tag a bunch of mediocre modern posts – I'd rather taggers start shifting their efforts towards improving tag descriptions (turn stubs into full fledged A or B tier tags [? · GW]), and thinking about how the tag ontology fits together (i.e. are some tags duplicates? which tags are related?)
My current guess is we should set the threshold to 40, and then I'm just going to strong upvote a bunch of older posts that deserve it to bump them over the threshold.
(Meanwhile, to all the users who have doing doing tons of tagging: thanks!)
Replies from: Raemon↑ comment by Raemon · 2020-08-18T21:35:05.593Z · LW(p) · GW(p)
Update: I didn't end up doing this, in part because it felt kinda lame to change the Progress Bar after the fact, partly because I realized it was relatively easy to at least get everything tagged with one of the core tags. (it's still nicer if posts can get More Specific Tags)
I would like to get the Progress Bar completed this week. Would anyone be interested in a zoom call coworking session where we set aside an afternoon or evening (maybe tomorrow if people are around) and then just... try to marathon the last chunk of tags?
(note that this isn't the last of the tagging work – the next chunks will probably be focusing more on "cleaning up the existing tag ontology, and making sure all the processed/approved tags get a one-paragraph description at least, and/or improving the text quality)
Replies from: KaynanK, Raemon, Raemon↑ comment by Multicore (KaynanK) · 2020-08-19T00:17:37.603Z · LW(p) · GW(p)
I would join such a zoom call. I have also been feeling the call of the progress bar's final phase.
Replies from: Rubycomment by Ruby · 2020-07-31T02:18:38.935Z · LW(p) · GW(p)
Someone mentioned that they thought the Concepts / Tag Portal was really nifty and they only just got round to looking at it, and that they thought it was motivating for tagging. I probably should have included a screenshot in the text (just added), but here's a comment with a larger one:
www.lesswrong.com/tags/all [? · GW]
comment by Multicore (KaynanK) · 2020-08-04T02:25:30.866Z · LW(p) · GW(p)
PSA: Voting on relevance is an important, underserved, and easy to contribute to area of the tagging system.
One person can create a tag, make a good description, and find a bunch of posts that fit it, but it takes multiple people's votes to create a decent ordering of posts from most to least relevant. Which posts are listed first will be an important part of the user experience.
This will be especially important for the more crowded tags, like the core tags, history, math, science, statistics, ai risk, and so forth.
Contributing can be as easy as just going through the list and upvoting posts that you've read and think are a good fit for the tag.
Edit: It would be nice to have a spreadsheet sorting tags by something like average relevance karma per post, to identify which tags most need votes.
Replies from: Raemon↑ comment by Raemon · 2020-08-04T19:52:14.408Z · LW(p) · GW(p)
Hey Multicore, I'd be interested in your thoughts on this alternate voting system [LW · GW] I had proposed awhile ago, which (among other goals) aims to shift things such that it's less necessary for multiple people to collaborate to vote on stuff.
The team has gone back and forth on whether it'd be an improvement, and/or whether it's enough of an improvement to be worth the dev-cost to switch.
Replies from: KaynanK↑ comment by Multicore (KaynanK) · 2020-08-04T20:54:56.837Z · LW(p) · GW(p)
I agree with most of your analysis in the comments (many downsides to karma, multiple choice has some advantages intuition-wise and makes it easier for a single user to make an ordering), but I thought of a couple more points. My mind seems to only be coming up with downsides of the multiple choice system, which might be because I'm prone to rationalizing why the status quo is good.
- Multiple choice has strategic voting implications too. If I think a 150 karma post and a 50 karma post are both "Top" relevance, but that the 50 karma post is better, I might rate the 150 karma post as "high" or lower.
- Multiple choice makes it harder to see where in the ordering your vote would make a post end up. Additionally, your vote either has no immediate effect or moves the post around by a lot, so a fine-grained adjustment is impossible. That might not necessarily be bad though, if post karma mattering is desired.
- If ordering is based only on the median vote, this makes it easy for a troll to vandalize a tag page even when the tagging system is mature. Just put the tag on a bunch of posts that don't already have it and rate them all "Top". With karma, the post order is more stable once a lot of people have voted. (This is the double edge of making it easy for a single person to have a big impact.)
However, these concerns balance out against the benefits you listed, so overall I don't have a strong opinion on which is better.
comment by Multicore (KaynanK) · 2020-07-29T18:47:25.231Z · LW(p) · GW(p)
A way I can contribute to the site without having to come up with brilliant original ideas? Excellent!
Replies from: DanielFilan↑ comment by DanielFilan · 2020-08-03T01:12:47.925Z · LW(p) · GW(p)
This is probably a joke, but in my experience, explaining other people's ideas is also a valued contribution if you explain it well and people are interested in the ideas.
Replies from: Rubycomment by Multicore (KaynanK) · 2020-08-22T16:19:17.931Z · LW(p) · GW(p)
The Progress Bar has been filled!
Dubious honor to https://www.lesswrong.com/posts/XCsKmLw8L55ZyYjnK/think-before-you-speak-and-signal-it [LW · GW] as the last post with 25+ karma to be tagged.
Replies from: Gyrodiot, Ruby↑ comment by Ruby · 2020-08-22T16:55:43.082Z · LW(p) · GW(p)
Woop! It's official. [LW · GW]
comment by Yoav Ravid · 2020-07-31T12:00:58.229Z · LW(p) · GW(p)
Making tags is fun!
Here are some tags i made:
Tagging [? · GW]
Open Problems [? · GW]
Meetups (topic) [? · GW]
Category Theory [? · GW](this one needs work because i know very little about category theory, but it seemed like a tag that should exist)
Urban Planning / Design [? · GW]
Replies from: adamShimi, Ruby↑ comment by adamShimi · 2020-07-31T14:55:34.541Z · LW(p) · GW(p)
Hey! I think it's cool that you created new tags. That being said, I do think that your description of category theory is not only a stub, but completely uninformative and dismissive about a mathematical field that has almost 70 years of work in it. I do think that explaining the controversy on the applicability of category theory is valuable, as we should question whether to use it for rationality and AI. But that should be a note at the end of the tag description, not the entire content of it.
(Note that I didn't change the tag description, because I don't want to force a change if I'm the only one thinking that. Maybe the point is only to describe how the word in the tag is used in LW, in which case the current tag description might work.)
Replies from: Yoav Ravid, Benito, Ruby↑ comment by Yoav Ravid · 2020-07-31T16:34:07.839Z · LW(p) · GW(p)
I agree with you, i struggled writing the description for it as i know very little about it, i just saw that it doesn't exist yet and went ahead creating it (maybe it would have been better if i left it with no description?). so i say go ahead and edit it, you'll surely do a better job than me :)
Replies from: adamShimi, Ruby↑ comment by adamShimi · 2020-08-01T12:07:58.572Z · LW(p) · GW(p)
I wrote a first version of the new tag description, I might rewrite some parts of it later. ;)
Replies from: Yoav Ravid↑ comment by Yoav Ravid · 2020-08-01T12:13:17.321Z · LW(p) · GW(p)
Thanks, much better :D
↑ comment by Ruby · 2020-07-31T19:21:14.423Z · LW(p) · GW(p)
+1 to be people creating tags they think are needed even if they're not sure about the description.
Replies from: Yoav Ravid↑ comment by Yoav Ravid · 2020-07-31T19:58:22.147Z · LW(p) · GW(p)
Thanks :)
In that case is it better to not write a description or write something knowing there's a high chance it'll be wrong (and leaving a note about it)?
↑ comment by Ben Pace (Benito) · 2020-07-31T20:19:28.346Z · LW(p) · GW(p)
I think it was the right call to write the tag descriptions that you did :)
Yeah, I'd continue to write something knowing there's a high change it'll be wrong.
↑ comment by Ben Pace (Benito) · 2020-07-31T17:37:56.092Z · LW(p) · GW(p)
ahahaha that tag description is hilarious (no offesne Yoav, thanks for making the tag!). adamShimi, please feel very welcome to change that tag description.
Replies from: Yoav Ravid↑ comment by Yoav Ravid · 2020-07-31T17:40:58.732Z · LW(p) · GW(p)
Well at least i made someone laugh :D
↑ comment by Ruby · 2020-07-31T19:28:36.173Z · LW(p) · GW(p)
As I've said a bit elsethread, I'm very in favor of 1) people creating tags they think should exist even if they don't know the topic well, 2) other people jumping in improving them where they see possible. I hope talk pages will make it easy to discuss any changes made, but most of the time I expect they'll just be all-round welcome improvements that don't need debating.
In short, if you think there's an improvement to be made, go for it! Even if you're not sure that anyone else agrees. Let them object after the fact.
More philosophical, I think we generally want to define terms like category theory in their proper standard usage. If the LW usage is off, we probably want to correct that or at least note the divergence in any description. (I don't remember if the S1/S2 description, but it would be good if it noted the difference between academic meaning and how it gets used around here).
If you end up writing an improved Category Theory description, I look forward to reading it.
comment by Bucky · 2020-08-04T16:18:38.324Z · LW(p) · GW(p)
Are there any thoughts on external links for tag wiki pages? I was looking at the social status tag, for example, and there are a few overcomingbias / ribbonfarm posts which I think would be useful but whether / how best to incorporate them isn't clear to me
Replies from: ChristianKl, Ruby↑ comment by ChristianKl · 2020-08-04T19:50:57.362Z · LW(p) · GW(p)
How about creating link posts for them?
Replies from: Raemon↑ comment by Ruby · 2020-08-04T16:57:51.531Z · LW(p) · GW(p)
My thought is they're great! I've generally hoped the descriptions-texts would contain more of them. I explicitly listed them as a factor in meriting a A-Class or B-Class tag-grade [? · GW].
I think a good way to do this is to have an "External Resources" section as part of the tag description that lists the external resources (with or without some context on them).
Replies from: Yoav Ravid↑ comment by Yoav Ravid · 2020-08-04T17:49:46.314Z · LW(p) · GW(p)
I wondered about that too since, for example, open problems in group rationality isn't on LessWrong, but is probably the main article on Group Rationality [? · GW].
Replies from: Rubycomment by Raemon · 2020-08-06T23:41:59.526Z · LW(p) · GW(p)
Quick note that there is now a Tagging Activity page available to everyone. You can see users who have tagged posts or upvoted/downvoted their relevance. (Corresponding reminder that tag activity isn't private)
comment by Gyrodiot · 2020-07-28T22:06:49.797Z · LW(p) · GW(p)
Thanks for the project, and the FAQ. I shall contribute.
Is there a way to retrieve the old tags from LW 1.0? I remember they were used to index Open Threads, for instance. I can't remember the details but that could be a good way to jumpstart some tags.
Replies from: Ruby↑ comment by Ruby · 2020-07-28T22:20:19.026Z · LW(p) · GW(p)
Awesome! Thanks! Don't hesitate hesitate to ask for any help making the process easier.
Re: LW1.0 tags -- good question. I believe they they were implemented via the old LW1.0 wiki and you can still view all those pages and their tagged posts. https://wiki.lesswrong.com/wiki/Special:AllPages
We've been planning to probably import that content, we estimate about 100 high quality articles/tags, but it's been waiting behind other dev work that seemed higher priority, e.g., Talk Pages for tags.
We've also got our eyes on the Arbital content, however that's a lot more work since we don't currently have all the features that Arbital uses.
comment by Raemon · 2020-08-05T21:04:53.031Z · LW(p) · GW(p)
I'm not sure the Newsletters tag quite makes sense. Generally if I'm interested in one Newsletter, what I want is more of that newsletter's topic (or, more of that Newsletter, specifically), rather than other random newsletters
I suppose it's still reasonably handy for finding a specific newsletter post that you half-remember.
Replies from: Yoav Ravid↑ comment by Yoav Ravid · 2020-08-05T21:47:11.301Z · LW(p) · GW(p)
I can see the usefulness of the tag. i think a way to solve the problem is to add links in the description to various newsletters and sequences that collect newsletters.
comment by Yoav Ravid · 2020-08-03T13:35:41.266Z · LW(p) · GW(p)
There now seems to be over a 100 Sequences [? · GW](depending on how one counts). I think it would be good to have tags for Sequences, Especially since they're not easily searchable through a search engine (or not that I'm aware of, at least).
comment by Yoav Ravid · 2020-08-06T07:06:35.012Z · LW(p) · GW(p)
PSA: You can now add and edit tags for a post from the drop down menu to the right of it (in places like the front page, tag pages, user profiles, etc.).
This makes it much easier to go through a big list of posts and adding relevant tags. you can go to one of the lists that are linked in the progress bar in the main page and start ploughing through the posts.
Thanks for the LW team for enabling this feature :)
Replies from: Ruby, Yoav Ravid↑ comment by Yoav Ravid · 2020-08-06T11:57:01.360Z · LW(p) · GW(p)
Just a small note, i think it would be nicer if the tag editing popup was higher, so when you hover over a tag you're able to see the description popup for it.
comment by Adam Zerner (adamzerner) · 2020-08-01T19:46:04.436Z · LW(p) · GW(p)
When I click "Add Tag", this is what I see:
Then I clicked to show more, because I know there are a lot more tags and want to make sure that if I tag a post it has all of the proper tags (because if I don't it'll be marked as tagged and it's likely that no one will return to it to add the proper tags):
But this view isn't organized well like the concepts portal is (below), so I felt the need to skim through each individual tag, which took a long time. Seems like it'd be a good idea to organize the above view to look more like the below view.
↑ comment by Raemon · 2020-08-01T19:56:34.144Z · LW(p) · GW(p)
Agreed. It's just actually a bit annoying to get it to work right, due to how the algolia search function works.
Replies from: Ruby↑ comment by Ruby · 2020-08-01T20:13:24.221Z · LW(p) · GW(p)
I think that adds to the reasons to perhaps replace Algolia.
Replies from: Raemon↑ comment by Raemon · 2020-08-01T20:42:56.138Z · LW(p) · GW(p)
I don't think this issue would go away with replacing algolia. The problem is a sort of generic "algolia is faster than making a database query, but that speed comes with some default settings that require re-wiring, which I'm sure is possible but requires some upfront costs that aren't part of my usual workflow. I think any search engine would come with the same issue."
Replies from: Ruby↑ comment by Ruby · 2020-08-01T20:52:30.096Z · LW(p) · GW(p)
(Don't know how much the rest of LW wants to hear our internal dev discussions, but) it's also things like Algolia, at least on our current plan, doesn't have personalization, e.g. to recommend tags a person previously used or tags we algorithmically guess would apply this post but would want a human you check.
Mostly going off Oli saying leaving Algolia is would be the way forward here. You might be right that no other solution will be better for this particular thing.
Replies from: habryka4↑ comment by habryka (habryka4) · 2020-08-01T21:25:47.792Z · LW(p) · GW(p)
I am pretty confused what any of this has to do with Algolia. The primary problem to me appears to be that we don't actually have a large fraction of the tags categorized in the tag hierarchy displayed on the All Tags page. We could show you a copy of the tag page table, but that would omit a lot of new tags, and also probably not be dense enough. We could develop some custom UI for that menu to group them, but that's mostly a bunch of work (and doesn't have super much to do with Algolia).
The site search will probably always have somewhat different constraints than normal database operations (in particular if we want to stay within the autocomplete paradigm), so I don't think anything about this would get easier if we switch away from Algolia (things like this are actually a domain where Algolia is pretty great).
Replies from: Ruby↑ comment by Ruby · 2020-08-01T22:28:14.092Z · LW(p) · GW(p)
I stand corrected and I hope Algolia is accepts my apologies for the slight. The actual table I don't think is much a possibility, if desirable at all, but structured things are good. The alternative is just ordered things, if we can accurately predict which things are likely.
Replies from: habryka4↑ comment by habryka (habryka4) · 2020-08-02T00:21:02.803Z · LW(p) · GW(p)
Yeah, ok. I do think personalization is blocked on Algolia, and I didn't really think about this as a potential solution to this (but it totally is). So yeah, maybe slighting Algolia was the right call.
comment by adamShimi · 2020-08-02T18:33:05.315Z · LW(p) · GW(p)
Just commenting to say it's pretty cool to see the bar filling up and the number of tagged posts growing up. Thanks to all the taggers!
Replies from: Ruby↑ comment by Ruby · 2020-08-02T21:29:52.293Z · LW(p) · GW(p)
Woop!
We haven't yet built any ways to recognize or reward the taggers, and I'd really like to. Any suggestions for how to do that?
Also, I don't want to shower a lot of attention on someone if they don't want it. We did say that tagging activity would be public, but I don't think that's salient yet. If you've tagged a lot and don't want a shout-out, please DM me before we get around to doing something proper.
One short-term idea: if you've been tagging a lot, you can comment here and I can tell you how many tags you've applied to date. Others could then upvote your comment if they wanted to, to say thanks. Also general intangible prestige.
Replies from: DanielFilan, adamShimi↑ comment by DanielFilan · 2020-08-03T01:35:38.107Z · LW(p) · GW(p)
We haven't yet built any ways to recognize or reward the taggers, and I'd really like to. Any suggestions for how to do that?
Publish a book of the best instances of people applying tags to posts in 2020.
↑ comment by adamShimi · 2020-08-03T18:37:15.685Z · LW(p) · GW(p)
Karma is nice. Maybe simply an appreciation post at some point, which could still not name people. Just let them know that they are appreciated.
I don't know if that's possible, but another option might be some sort of "rank" or "badge" for top taggers. That being said, one might ask why have ranks only for this specific case, and not in general.
comment by Yoav Ravid · 2020-07-31T06:21:41.763Z · LW(p) · GW(p)
The progress bar in the main page is super cool. I'm curious, how is it being calculated?
Edit: oh, when you hover over it it says "X out of 4735 posts have been tagged (filtered for 25+ karma)"
So i guess that's how
↑ comment by Yoav Ravid · 2020-07-31T16:43:26.542Z · LW(p) · GW(p)
Is it just me or is the number of posts tagged that's shown on hover is going down with time?
Replies from: habryka4↑ comment by habryka (habryka4) · 2020-07-31T17:56:34.751Z · LW(p) · GW(p)
Ahh, hmm. That is embarrassing. Hmm, I wish I had a better excuse for this. Hmm...
I mean, look over there a three-headed monkey!
(Will be fixed within the hour)
Edit: And it's fixed. Sorry about that!
comment by Multicore (KaynanK) · 2020-08-07T00:41:58.338Z · LW(p) · GW(p)
I notice that when you try to tag a post but its relevance was in the negatives and your vote doesn't bring it above zero, the site doesn't give any feedback. It looks like there was a bug, or your connection messed up and didn't submit the tag properly.
This is somewhat mitigated now by being able to look at the tag voting page to confirm that you vote went through, but a lot of people won't know to do that.
Replies from: Raemoncomment by Raemon · 2020-08-05T21:03:37.940Z · LW(p) · GW(p)
I'm not sure how to distinguish between the Empiricism tag and the Philosophy of Science tag. Anyone have a strong sense of what goes it one vs the other?
Replies from: KaynanK, Raemon, Yoav Ravid↑ comment by Multicore (KaynanK) · 2020-08-05T21:29:07.163Z · LW(p) · GW(p)
My sense is that Empiricism is specifically about experimentation and making beliefs pay rent in anticipated experiences, while a lot of the posts in Science are about academia and the social institutions of science.
There's some overlap between them. Additionally, Empiricism overlaps with Anticipated Experiences and Science overlaps with Replication Crisis.
Empiricism was added more recently. I went back and tagged some of the posts in Science as Empiricism.
Replies from: Kaj_Sotala↑ comment by Kaj_Sotala · 2020-08-06T15:36:52.869Z · LW(p) · GW(p)
I agree with this; I wouldn't expect Empiricism to have posts like e.g. Is Science Slowing Down? [LW · GW] or Ed Boyden on the State of Science [LW · GW].
↑ comment by Raemon · 2020-08-06T17:13:58.100Z · LW(p) · GW(p)
I guess slightly new question: is Empiricism a strict subset of Philosophy of Science? Is there an example of an Empiricism post that isn't in Philosophy of Science?
Replies from: Yoav Ravid↑ comment by Yoav Ravid · 2020-08-06T18:31:25.995Z · LW(p) · GW(p)
Maybe the "practical" side of empiricism.
↑ comment by Yoav Ravid · 2020-08-05T21:45:20.857Z · LW(p) · GW(p)
Hey, i added Empiricism. It seemed fitting as it is a frequent topic on LessWrong (surprise surprise :), it's more specific than Philosophy of science (which currently has 63 posts), and is also named as one of the virtues of rationality.
comment by Raemon · 2020-08-02T18:15:41.414Z · LW(p) · GW(p)
Tagging meta question:
There's a Calibration (Probability) tag. How important is it to keep that distinct from other forms of calibration (i.e. if you think some parameter will be within particular bounds, those bounds tend to be correct).
Prompted by this post on time calibration.
I suppose that all calibration is implicit probability calibration (i.e. if I think something will take between 15 and 45 minutes, I'm sort of implicitly claiming it has a high probability of being so, even if I didn't concretely decide it was my 90% confidence interval). But, if all calibration can be reformulated as probability, do we need the "(probability)" disambiguation?
Replies from: Ruby↑ comment by Ruby · 2020-08-02T18:58:33.133Z · LW(p) · GW(p)
Not all calibration is probability calibration, e.g., calibrating my scales or voltmeter, but as you suggest, calibration discussion on LessWrong is effectively calibration about credences/probabilities. Not worth keeping finer gradations distinct.
But I think the disambiguation is good because it explains the tag to someone new on LessWrong and doesn't rely on your already knowing the content, so I'm in favor of the disambiguation. The way we use calibration is a our own jargon, so good to explain a bit what we mean just in the title.
Replies from: Raemon↑ comment by Raemon · 2020-08-02T19:39:33.136Z · LW(p) · GW(p)
I can see the case for that, but FYI it just made me go make this meta comment rather than intuitively classifying a calibration post. I think it might be fine to have the disambiguation live in the text rather than the title.
Replies from: Ruby, Kaj_Sotala↑ comment by Ruby · 2020-08-02T21:10:49.300Z · LW(p) · GW(p)
Epistemic status: I'm generally pro disambiguations in parentheses, I was the one who advocated we borrow the practice from Wikipedia. I'm really not sure in the case between Calibration and Calibration (Probability), so I'm just trying to think through this with others.
The tag description is this:
Do the events that you give a 70% probability in advance, actually end up happening 70% of the time?
It's pretty brief, I'm guessing that didn't clarify enough. I actually suspect here that if the description had been like the following, clarification wouldn't have been necessary. Raemon?
Someone is probability or credence calibrated if the things they predict with 70% chance of happening in fact occur 70% of the time. Importantly, calibration is not the same as accuracy. Calibration is about accurately assessing how good your predictions are, not making good predictions. Person A, whose predictions are marginally better than chance, e.g. 60% of them come true, and who knows that, is well-calibrated. In contrast, Person B, whose predictions are 90% accurate, yet thinks they are 99% accurate, is more accurate than Person A while being less well calibrated.
Knowing how good your predictions are is a key rationalist skill. Among other things, being calibrated lets you make good bets [Link to Betting tag]/make good decisions [link Planning & Decision-making tag], communicate information helpfully to others if they know you to be well-calibrated [link to Group Rationality], and helps prioritize which information is worth acquring [link to VoI tag].
Note that calibration applies to all expressions of quantified confidence in beliefs/predictions [reference: Anticipate Experiences]. For example, calibration applies to whether a person's 95% confidence intervals capture things 95% of the time. Or if their 80% chance of completion estimates are met 80% of the time (not much more or less). Trivially, odds ratio placed on things are convertible to probabilities.
See also: prediction & forecasting
I think this makes it much clearer that "tends to be correct" is always a quantitive/probabilistic statement beneath the hood.
Hmm, what this makes me think is really it's about calibrating credences, which isn't standard jargon anyway. So maybe just plain "Calibration" is better than "Probability". Or maybe Calibration (belief strength)?
My main thought now is the issue wasn't the name so much as lack of good explanation for the topic. [No complaint against the tag creator – it was a good tag to make.] I'd kind of like to have a big tag description writing push sometime after the plain "tag at all" push since so far few tags have good explanations. But we'd have to decide we're definitely moving more in this wiki-ish direction.
Yeah, I'd propose Calibration or Calibration (belief-strength).
I dislike Probability Calibration because I dislike leading adjectives/modifiers and prefer the main thing to be the first word in the noun phrase (some languages like Hebrew do this). I expect people to be looking for the core thing, e.g. Relationships, "R", and if you put modifiers in front, e.g. "Business", "Personal/Interpersonal", "Romantic", "Conceptual", you then require someone to guess which modifier you used, and also split up Relationship tags from being adjacent in an alphabetical list.
I think having as much in the title as possible is better than in the text, just because even triggering the hover-over is 10-20x costly than just skimming all the titles, and also doesn't work on mobile where are no hover-overs. I think if someone's looking over the tags list and sees "Calibration (belief-strength)" they have much more of an idea of what tag is about than just Calibration which is pretty opaque to an outsider.
↑ comment by Kaj_Sotala · 2020-08-02T21:55:29.873Z · LW(p) · GW(p)
I dislike Probability Calibration because I dislike leading adjectives/modifiers and prefer the main thing to be the first word in the noun phrase (some languages like Hebrew do this). I expect people to be looking for the core thing, e.g. Relationships, "R", and if you put modifiers in front, e.g. "Business", "Personal/Interpersonal", "Romantic", "Conceptual", you then require someone to guess which modifier you used, and also split up Relationship tags from being adjacent in an alphabetical list.
There is that, but at the same time, you probably wouldn't want tags like Experiences (Anticipated) or Induction (Solomonoff). I don't have any principled argument for this, but to me "Probability Calibration" feels more like one of those examples. It being put alphabetically close to "Probability" may also be good.
(I also keep feeling confused by Relationships (Interpersonal) each time I see it, though that's probably in part because there's no other 'Relationships', so I just think 'well what other relationships could you even mean' and then don't find another Relationships that it would be contrasted with.)
Replies from: Ruby↑ comment by Ruby · 2020-08-02T22:25:14.703Z · LW(p) · GW(p)
That's a fair point to raise. I think it's more work to give an explicit theory for why those are different. Something like those are both technical terms/jargon and they don't belong to a broader class of things that we might be discussing. In a world where there were three types of induction we discussed, might then go for Induction (Solomonoff). Also that they're the actual phrase people use. I don't think I can remember anyone saying "probability calibration" ever.
With Relationships (Interpersonal), I think it makes sense because to me, the default way to read "relationships" is specifically romantic relationships. Like if your friend says "I'm reading a book about relationships", what do you assume? To make it clear the tag also covers friendship, family, and work relationships I think actually does require some disambiguation even if the site doesn't have any other relationship tags right now.
↑ comment by Yoav Ravid · 2020-08-03T05:27:00.042Z · LW(p) · GW(p)
So perhaps we should also have Relationships (Romance)? :)
Replies from: Ruby↑ comment by Yoav Ravid · 2020-08-03T06:16:12.342Z · LW(p) · GW(p)
I went ahead and edited the tag to have your description
Replies from: Ruby↑ comment by Kaj_Sotala · 2020-08-02T20:25:04.304Z · LW(p) · GW(p)
"Probability Calibration" rather than "Calibration (Probability)" feels like a more natural name for the tag, while keeping the disambiguation.
Replies from: Rubycomment by Yoav Ravid · 2020-08-01T10:21:54.799Z · LW(p) · GW(p)
Would be nice to be able to see tags by date created, or something to know what new tags are created, so we can take a look and think if there's anything that needs to go there.
Replies from: Ruby↑ comment by Ruby · 2020-08-01T17:33:04.682Z · LW(p) · GW(p)
That makes sense! That's the kind of thing we should build into a proper tagging dashboard.
This spreadsheet has tags sorted by last changed (edited or posts added) but I'll need to add in an additional column in order to sort by creation date. I can get around to it soon, if it's helpful.
comment by Gunnar_Zarncke · 2020-07-31T00:07:28.880Z · LW(p) · GW(p)
I have tagged a few posts from the top of the spreadsheet but not too many because it caused me reading too many old posts...
I have added the tag Habits; hope that makes sense. I'm not too clear about the taxonomy.
How frequently is the spreadsheet updated? UPDATE: Every 5 Min according to the OP.
Replies from: Ruby, Raemon↑ comment by Raemon · 2020-07-31T00:26:38.600Z · LW(p) · GW(p)
Thanks! Yesterday I was looking at the Martial Art of Rationality and trying to figure out what to tag it, and kinda gave up. I believe you created the tag for that? (I think it was the first tagged post). Kudos, it was a good solution I think.
comment by ESRogs · 2020-07-29T19:18:36.024Z · LW(p) · GW(p)
We'd like to build a small community around taggers – the people who maintain the ontology of LessWrong's library ensuring that desired information can always be found.
Maybe this is a dumb question but, is this actually needed?
Can we get what we want with people just randomly adding tags when they notice? Do we need to have people specializing on this?
I'd expect that a bunch of work would be needed up front to get the tag system into a good state, but I'd think most of that work has been done already (by the LW team, and others). And then going forward I'd expect much less work to be required. Am I missing something?
Replies from: habryka4↑ comment by habryka (habryka4) · 2020-07-29T19:34:53.997Z · LW(p) · GW(p)
I think tagging is actually pretty hard. Like, by default you get a ton of synonyms of the same concepts, and there aren't good redirects, and the tags don't have good descriptions, and there is lots of ambiguity, and when someone creates a new tag old posts don't reliably get tagged. Our tagging system is also more similar to being a wiki, and in-general my research into wikis suggests that basically all functional ones are maintained by a relatively small group of highly dedicated editors, and that it generally doesn't work to just have everyone randomly edit and add things.
Replies from: ESRogscomment by Kaj_Sotala · 2020-07-29T13:24:11.855Z · LW(p) · GW(p)
Alternatively, we have an automatically updating spreadsheet (every 5 min) that tracks the tags on the most viewed posts according to our data and their current tags.
Note that this spreadsheet seems to open by default to the "sorted by karma" tab, and you have to manually switch to the "sorted by view rank" tab (spent half a minute wondering whether the link was incorrect before happening to look at the tab list).
(Also: Oh wow, our most viewed post [? · GW] in one with four karma?)
Replies from: habryka4↑ comment by habryka (habryka4) · 2020-07-29T17:10:03.375Z · LW(p) · GW(p)
Note: This is most viewed for the last 30 days I think. That post in particular tends to show up at the top every few months whenever you get closer to U.S. College application periods.
And yeah, sometimes the posts that get a ton of views really aren’t very good. For a while one of our most viewed post was one at -4 karma called “The Effects of Religion [Draft]”. It‘s been one of the reasons why I’ve been hesitant to include views as an easily accessible metric on the site, because I know how frequently it diverges from quality.
Replies from: Rubycomment by ChristianKl · 2020-08-14T20:25:50.604Z · LW(p) · GW(p)
In https://www.lesswrong.com/tag/intuition?showPostCount=true&useTagName=true the description seemed to be copied from Wikipedia but the content is not licensed in a way that's compatible with Wikipedia's license.
Replies from: habryka4, Ruby↑ comment by habryka (habryka4) · 2020-08-19T19:48:24.395Z · LW(p) · GW(p)
I added the above notice about licensing requirements, which I think should make this fine.
↑ comment by Ruby · 2020-08-16T23:19:26.725Z · LW(p) · GW(p)
Thanks for flagging this. I think we just have to add attribution? Will work on it.
Replies from: ChristianKl↑ comment by ChristianKl · 2020-08-17T09:48:10.480Z · LW(p) · GW(p)
It's a share-alike license, so attribution alone is not enough.
Replies from: Ruby↑ comment by Ruby · 2020-08-19T19:50:24.114Z · LW(p) · GW(p)
Habryka (who has a bunch of copyright knowledge) thinks that appending the following is enough:
(From Wikipedia 1, 2, this section is licensed as CC BY-SA 3.0 Unported License to conform with the Wikipedia license requirements)
comment by Multicore (KaynanK) · 2020-08-14T19:49:50.686Z · LW(p) · GW(p)
lesswrong.com/tagvoting isn't loading for me. All I get is the error " Error: TypeError: Cannot read property 'isEvent' of null". I'm using Chrome, if that's relevant.
comment by Yoav Ravid · 2020-08-03T13:32:08.325Z · LW(p) · GW(p)
I wanted to create a 'Law and Legal Systems' tag, but then i saw there's a Government [? · GW] tag, currently only with 2 posts. should these tags be separate, or should i change the current tag to "Law, Legal Systems And Government"?
Replies from: Rubycomment by Multicore (KaynanK) · 2020-07-31T19:52:31.861Z · LW(p) · GW(p)
The tags page is occasionally suddenly replacing itself with the message "Error: TypeError: Cannot read property '_id' of null", forcing me to reload the page. Has anyone else seen this?
Edit: I also got the same error on the page for a post, when I added a tag, the server response was slow, and I tried to add it again.
Replies from: Ruby, Yoav Ravid↑ comment by Yoav Ravid · 2020-07-31T20:24:45.922Z · LW(p) · GW(p)
Happened to me too (though perhaps a slightly different error message, not sure)
comment by ESRogs · 2020-07-29T19:05:27.668Z · LW(p) · GW(p)
It's a good to start by becoming familiar with LessWrong's existing tags. You can see them on the new Concepts page.
In the text, Concepts shows up green, like it's supposed to be a link, but it doesn't go anywhere. Was it supposed to be a link?
Replies from: ESRogs↑ comment by ESRogs · 2020-07-29T19:06:37.100Z · LW(p) · GW(p)
Also, maybe it was supposed to be "It's a good idea to start..."?
Replies from: Ruby↑ comment by Ruby · 2020-07-29T19:10:46.161Z · LW(p) · GW(p)
Thanks, fixed on both accounts!
Replies from: ESRogs↑ comment by ESRogs · 2020-07-29T21:54:08.030Z · LW(p) · GW(p)
The Concepts link is still not going anywhere for me :-/. When I inspect on Chrome, it shows up like this:
↑ comment by habryka (habryka4) · 2020-07-29T22:28:40.033Z · LW(p) · GW(p)
Now actually fixed (there was a typo in the URL).
comment by adamShimi · 2020-08-16T16:23:46.189Z · LW(p) · GW(p)
Just realized when I tried to tag this post [LW · GW] that there isn't an Abstraction tag. Should there be one, or is there an equivalent tag?
Replies from: Rubycomment by Multicore (KaynanK) · 2020-08-02T18:19:45.758Z · LW(p) · GW(p)
Diseased disciplines: the strange case of the inverted chart [LW · GW] is an interesting case because tagging it correctly feels like a spoiler.
Replies from: Yoav Ravid↑ comment by Yoav Ravid · 2020-08-02T20:14:23.733Z · LW(p) · GW(p)
After i read your comment i decided to read the post. i saw the tag programing*, but forgot about it midway and it didn't spoil the surprise for me - still i think you're probably right it would for other people.
Anyway, i also added it to information cascades, as it seems relevant and doesn't spoil it.
*not sure how to do a spoiler tag
comment by Yoav Ravid · 2020-07-31T09:54:05.096Z · LW(p) · GW(p)
Would be nice to be able to add tags from the drop-down menu to the right of posts (in places like the homepage and users profiles). This would speed up the process by a lot.
Replies from: Ruby↑ comment by Ruby · 2020-08-01T01:46:44.424Z · LW(p) · GW(p)
I agree. Currently, there's the "Edit Tags" button that gets you there, but we've been meaning to replace that with the "AddTag" button directly. Thanks for the feedback.
Replies from: Raemon