The Wiki is Dead, Long Live the Wiki! [help wanted]

post by Ruby · 2020-09-12T03:34:50.622Z · LW · GW · 32 comments

Contents

    Quick Links
    The Three Import Types
  Join the Tagger Slack!!
    Join the Tagger Slack here
  More Details on Processing Wiki Pages
    Merging pages
    Optimizing the opening paragraph
    Updating Pages
    Tagging Relevant Posts
  Conclusion
None
32 comments
That's just a few of them. We imported like 5x as many as these.

With the goal of eventually archiving it fully, we have imported 573 pages and 266,000 words of content from the old LessWrong wiki to LessWrong 2.0

The old wiki is a great store of knowledge and still gets two thousand pageviews each day. Incorporating it into the new site gets us at least the following benefits:

The Three Import Types

Pages have been imported in one of three ways:

  1. 76 are imported as new tags that can be applied to posts.
  2. 111 are merged with existing tag pages, which is currently in progress and could use some help (see below).
  3. 386 are imported as "wiki-only" pages . These pages cannot be applied to posts and do not currently appear on the Concepts page.

 

The list of imported of all 573 wiki pages [? · GW]

 

To be honest, it would be more accurate to say that we are part-way through the import. We have completed the programmatic part, and now there remains some manual work to do, hence the help needed.

First, there is some general clean-up of links and other elements that didn't import correctly. Second, and more importantly, a manual text merge is required for the 111 pages are being merged into existing tags. This means taking the text of the existing tag (if it has any) and combining it appropriately with the old wiki page.

Right now, "merged pages" have the old pages' revision history (click History on the tag), but the current text is unchanged.

You can help us out fixing up the wiki import and follow along on completed/incomplete work you can find on the Tagging/Wiki Dashboard. More on how to help below, though the hover-overs on the tag flags.

 

The New Tagging Dashboard

Join the Tagger Slack!!

A couple of weeks ago we created a Slack workspace for dedicated taggers to be able to discuss tagging issues and talk directly to the LessWrong team about it. Following initial success plus good timing with the wiki import campaign, we're opening that Slack to anyone who wants to help with tagging.
 

Join the Tagger Slack here

You can also still leave comments on the Tagging Open Call / Discussion Thread [LW · GW].

More Details on Processing Wiki Pages

Here is a more detailed list of the kinds of work to be done:

Merging pages

Optimizing the opening paragraph

On LessWrong 2.0 (this site), the opening paragraph is what shows on hover-preview for tags, making it very important. It's worth optimizing the opening paragraph of imported pages.

Updating Pages

Tagging Relevant Posts

Conclusion

I'm excited to have the great content from the old LW wiki now incorporated into the new site, in many ways, it's long overdue. 

Thanks to everyone in advance who helps us complete the import!

32 comments

Comments sorted by top scores.

comment by Chris_Leong · 2020-09-13T04:11:41.390Z · LW(p) · GW(p)

So I'm confused - where can we access the wiki now? You said some were classed as tags and others weren't? So are *all* the articles under concepts; or only some? And how do we create a new wiki page?

Replies from: Ruby
comment by Ruby · 2020-09-13T05:27:18.814Z · LW(p) · GW(p)

I was expecting people to be confused, so I’m glad you asked. It’s currently a slightly messy situation while we still figure out some design decisions and  possibly build a few missing pieces. 

  • The only wiki pages right now are those which have been imported. You can find them all on this page. [? · GW]
  • Right now, both tags and wikis are implemented as Tags, except some of them have a flag that deactivates the tag-specific features and so makes it a pure wiki page. Only admins can set this flag right now.
  • Wiki-only pages currently do not appear on the Concepts page. There are too many for it currently,  we will have to change somehow.
  • Currently, users cannot create new wiki pages. We will change that, but there are still some details to be decided on how to actually best go about. For now, you can make tag pages and we’ll change them to be wiki-only if that makes more sense (and ping us you want to bring to our attention that a tag should be a wiki).
  • eventually, wiki pages will be findable in search and in other parts of the site.

Overall, the design we’ve ended up with combines tags and wiki pages as almost the same thing, except tags can be applied to posts. We’ve considered the name “twiki“.

While we’re not done building the system, it’s a bit messy. It still seemed good to import the old wiki at this point though. Hopefully we’ll get everything else to a not confusing state soon.

Replies from: Dreaded_Anomaly, RobbBB, ChristianKl
comment by Dreaded_Anomaly · 2020-09-17T02:53:01.788Z · LW(p) · GW(p)

"Twiki" is already the name of a wiki-related product (https://twiki.org/), so that might be confusing.

Replies from: Ruby
comment by Ruby · 2020-09-17T20:03:16.119Z · LW(p) · GW(p)

That project is Twiki, not twiki like we're suggesting. Shouldn't be any confusion.

Replies from: digital_carver
comment by digital_carver · 2020-09-21T06:56:54.923Z · LW(p) · GW(p)

Shouldn't be any confusion.

Famous last words.

(I don't think it matters enough to avoid that name, but there will certainly be small confusions, by search engines and by people familiar with that Twiki and getting into LessWrong as newcomers. I can't imagine that leading to any particular negative consequence other than some little wasted time and brainpower for those, though.)

Replies from: habryka4
comment by habryka (habryka4) · 2020-09-21T17:13:46.199Z · LW(p) · GW(p)

(Just to be clear, I understood Ruby's comment to be a joke)

comment by ChristianKl · 2020-09-22T09:02:25.710Z · LW(p) · GW(p)

I might be too much in the German way of nanimg things but tagwiki seem superior to twiki. Twiki is unclear to pronounce (do you say it as T wiki or TW iki?) and also not explainatory to someone who doesn't understand it. 

Edit: StackExchange seems to use "tag wiki" https://meta.stackexchange.com/questions/214337/what-is-a-tag-wiki-how-do-i-write-a-good-one It seems that how things have to work in English.

Replies from: Ruby
comment by Ruby · 2020-09-22T15:57:46.307Z · LW(p) · GW(p)

I think the natural pronounciation in English is twik-ee, rhymes with quick-ee. If it were T-wiki, you'd spell it like that.

Originally I was calling them wiki-tags, till someone jokingly called them twiki's which just seemed pretty good - shorter. 

You're right about the cost of the meaning not being obvious. 

Probably should poll people.

Replies from: Ruby, Ruby, Ruby, Ruby, Ruby
comment by Ruby · 2020-09-22T15:59:50.770Z · LW(p) · GW(p)

Upvote this quote for "wiki-tag"

comment by Ruby · 2020-09-22T15:58:46.832Z · LW(p) · GW(p)

Upvote this comment for "tag-wiki"

comment by Ruby · 2020-09-22T15:59:19.426Z · LW(p) · GW(p)

Upvote this comment for "twiki"

comment by Ruby · 2020-09-22T16:07:50.261Z · LW(p) · GW(p)

Upvote this comment for "twig"

comment by Ruby · 2020-09-22T16:03:18.345Z · LW(p) · GW(p)

Downvote this comment to cancel out me getting karma for running a poll.

Also I think voting for more than 1 option is good if you're indifferent between two options.

I think both "karma" and "number of votes" are both interesting. Also the LW team might decide some consideration, so this can strongly influence our thinking but not completely determine it, necessarily.

comment by Gyrodiot · 2020-09-14T18:57:02.769Z · LW(p) · GW(p)

Thank you for the import.

Once again, the Progress Bar shall advance. It will probably be slower this time. No matter: I shall contribute.

comment by Ben Pace (Benito) · 2020-09-12T07:02:46.840Z · LW(p) · GW(p)
  • Pages imported from the old wiki now appear in search results on LessWrong proper.
  • Pages imported from the old wiki benefit from all the features of new LessWrong such as hover-preview, subscriptions, commenting, and functioning as tags on posts.
  • Since LessWrong proper is an active site, hopefully, the wiki content continues to get updated.
  • People who land on the old wiki content will more easily find the rest of the awesome content/activity/community on LessWrong proper.
  • LessWrong proper becomes more like the proper LessWrong that it was meant to be, unlike the wiki which has far less activity than is proper for LessWrong proper.

I admit, I added the last bullet ;)

comment by totallybogus · 2020-09-23T04:18:25.413Z · LW(p) · GW(p)

I can still identify a few pages on the old wiki that seem to have no matching entity in the new "tagging" system, e.g. Adversarial process (a general, widely-used notion wrt. which the rationalist Adversarial collaboration may be a special case -- so it seems like a fairly important thing to have!). Will these pages be imported in the future?

Replies from: Ruby, habryka4
comment by Ruby · 2020-09-23T05:43:06.011Z · LW(p) · GW(p)

It was indeed I who went through most of the old wiki pages and decided what to do with them. There where ~600,  so I do expect to have made some mistakes, and would very happy to discuss if I missed any valuable ones. 

Looking at Adversial process, I don't see why I wouldn't have imported it. And yet I didn't mark it anything on my spreadsheet, so my bad:
 

We can import it. Let me know any others you think should be there. 

comment by habryka (habryka4) · 2020-09-23T05:04:11.658Z · LW(p) · GW(p)

Ruby went through all the pages and decided whether to import them or not. I think it's unlikely we are going to import most of the remaining pages (some of which were pretty random and low-quality), but we will make sure they stay accessible, and if there is any individual post that isn't covered by the import that you feel is missing, there is a good chance we can just add it to the import. Which specific ones we should import is Ruby's call. 

Replies from: totallybogus
comment by totallybogus · 2020-09-23T05:27:33.950Z · LW(p) · GW(p)

Thamks for that clarification! I think it would be OK to discuss the merits of importing any given page, perhaps in this very LW thread. Separately, there is quite a bit of Wiki content that's now been 'hidden' in the new system as a result of being merged with an existing tag, and the more "in-depth" portions of that content, if considered worthwhile, should probably be moved to newly-created 'wiki-only' pages, so as to reduce confusion among users who only care about the bare "tagging" aspect.

(I have in mind, e.g. the discussion of problematic 'persuasion' technology in the Dark Arts wiki page, or the 'community' conceptual metaphor for computer-mediated communication as discussed in the page on "Groupthink". That kind of content can make sense on a "wiki only" page, not so much in the bare description of a "tag"!)

Replies from: Ruby
comment by Ruby · 2020-09-23T05:48:31.541Z · LW(p) · GW(p)

The vision is that tag pages should be wiki pages, no matter the depth. (Long pages get displayed with truncation on load, the rest behind "Read More", so it's fine). I think it's actually good to keep the longer discussion on the one page for the topic.

I suspect that most of the "missing content" comes from the fact we haven't finished "merging" the old wiki pages with existing tags, and therefore the current text is just whatever the new tag already had. (And the revision/history reviewer makes it seem like this intentional, but it's not.)

Merging = combine new and old text in whatever most makes sense. Combine and take whichever bits are better when they conflict.

The campaign to get through all the manual import processing continues! We just launched the new tagging dashboard today, on which you can filter for pages requiring merging [? · GW]. Currently 75 remaining for merges.. 

Replies from: Ruby
comment by Ruby · 2020-09-23T05:50:12.252Z · LW(p) · GW(p)

Ah, yep, Dark Arts [? · GW] has the flag set for requiring a manual merge. These show up on the tag dashboard and when you go to edit the tag. Someone will get to it! (Of course, feel free to be someone, but no pressure.)
 

comment by Joshua_Fox · 2020-09-17T11:41:50.966Z · LW(p) · GW(p)

Wonderful project! I enjoyed writing for that old wiki.

Will you redirect given pages from the old to the new wiki, or at least have a link that readers can follow?

Replies from: habryka4
comment by habryka (habryka4) · 2020-09-17T17:59:14.538Z · LW(p) · GW(p)

Yep, after we are done with the import, we are going to redirect all the pages we imported. And then probably make all the remaining pages on the old wiki read-only, so we don't have to maintain a whole separate wiki system forever. 

Replies from: Ruby
comment by Ruby · 2020-09-17T18:08:06.298Z · LW(p) · GW(p)

Bellroy/Trike (who hosts/maintains the old wiki) has already set a testing server for the redirects and they appear to be working perfectly.

comment by Pongo · 2020-09-12T23:28:43.723Z · LW(p) · GW(p)

I expect "x imported out of y", or "x imported, y remain" to be more motivating than the current "y remain" on the import progress bar.

Replies from: habryka4
comment by habryka (habryka4) · 2020-09-12T23:35:44.549Z · LW(p) · GW(p)

Yeah, seems reasonable. We show the full number on hover-over, but seems probably good to also display the full number by default.

comment by riceissa · 2021-01-17T21:39:24.117Z · LW(p) · GW(p)

The Slack invite link seems to have expired. Is there a new one I can use?

Replies from: Raemon
comment by Raemon · 2021-01-17T23:12:47.108Z · LW(p) · GW(p)

Here's a new one. I'm not sure how to make a longterm one.

https://join.slack.com/t/lwtaggers/shared_invite/zt-ky9ys5dx-MOxZiySdd5lX7XTvBde78Q

Replies from: riceissa
comment by riceissa · 2021-01-18T03:12:56.350Z · LW(p) · GW(p)

Thanks!

comment by Tofly · 2020-09-16T19:11:53.240Z · LW(p) · GW(p)

For wiki pages which are now tags, should we remove linked LessWrong posts, since they are likely listed below?

What should the convention be for linking to people's names? For example, I have seen the following:

  • LessWrong profile
  • Personal website/blog
  • Wiki/tag page on person
  • Wikipedia article on person
  • No link
  • No name

Finally, should the "see also" section be a comma-separated list after the first paragraph, or a bulleted list at the end of the page?

Replies from: Ruby
comment by Ruby · 2020-09-16T19:41:43.656Z · LW(p) · GW(p)

Great questions! I'll answer with what I think, and people can argue with me if they want about what the conventions should be.

For wiki pages which are now tags, should we remove linked LessWrong posts, since they are likely listed below?

If they seem like not especially top posts for the tag, then yes.

If the list seems like it is really top posts for the tag, then seems good to keep them. By top I mean "introduces the term" or "is really good explanation for it".

 

What should the convention be for linking to people's names? For example, I have seen the following:

  • LessWrong profile
  • Personal website/blog
  • Wiki/tag page on person
  • Wikipedia article on person
  • No link
  • No name

For a LessWrong user with a modern LessWrong account, I think we should link to their current profile page, especially if it's currently linking to a wiki page that's not very good. 

If there's current a link to a wiki page for the person and it has a bunch of details, I think that should be kept, but we should update the wiki profile page to have a link to their LessWrong profile.

If there's a link to a Wikipedia page (e.g. Kurzweil) or a personal blog, I think that's good to leave as is.

If there's no link, that's fine to leave as is for the purpose of this import edit, but supererogatory to add a link to elsewhere.

No name seems fine if there's a link to a post. You can see the name via hover-over.

Finally, should the "see also" section be a comma-separated list after the first paragraph, or a bulleted list at the end of the page?

I personally think that the comma-separated list near the top is very valuable for helping quickly discover adjacent topics they might be interested in.

My thought is: there should be a comma-separated list near the top for especially related tags, wikis, and maybe sequences.

If there's additional stuff, e.g., links to external links to other resources, posts, whatever, those should be in a separate list further down on the page.

I'm actually unsure of what the title of those sections should be when there are two. Seems weird to have them both be "see also", so sometimes I've used "related tags" and I saw someone else write "related tags and wikis" which also seems fine. I'm interested in suggestions.