Recommendation Features on LessWrong

post by habryka (habryka4), jimrandomh · 2019-06-15T00:23:18.102Z · LW · GW · 36 comments

Contents

  Continue Reading
  From the Archives
  Why These Features
    LessWrong as a Repository for "long content"
    LessWrong as a Nudge
  Caveat: Addictiveness
  These Are Beta
None
36 comments

Today, we're rolling out several new beta features on the home page, which display recommended posts to read. The first is a Continue Reading section: if you start reading a multi-post sequence, it will suggest that you read another post from that sequence. The second is a From the Archives section, which recommends highly-rated posts that you haven't read, from all of LessWrong's history.

To use these features, please ensure you are logged-in [? · GW].

Continue Reading

Sequences are a mechanism on LessWrong for organizing collections of related posts. Anyone can create a Sequence from the library page [? · GW]. If you write a series of posts and add them to a Sequence, they will have Previous and Next links at the top and bottom; if you create a Sequence out of posts by other authors, they will have Previous and Next links for readers who came to them via the Sequence.

When you are logged in and read a post from a Sequence, the first unread post from that sequence will be added as a recommendation in the Continue Reading section, like this:

If you decide not to finish, hover over the recommendation and click the X button to dismiss the recommendation. For logged-out users, the Continue Reading section is replaced with a Core Reading section, which suggests the first posts of Rationality: A-Z [? · GW], The Codex [? · GW], and Harry Potter and the Methods of Rationality [? · GW].

From the Archives

The home page now has a From the Archives section, which displays three posts randomly selected from the entire history of LessWrong. Currently, a post can appear in this section if:

(1) you've never read it while logged in (including on old-LessWrong),

(2) it has a score of at least 50, and

(3) it is not in the Meta section, or manually excluded from recommendation by moderators. (We manually exclude posts if they aged poorly in a way that wouldn't be captured by votes at the time -- for example, announcements of conferences that have already happened, and reporting of studies that later failed to replicate.)

Currently, if a post is eligible to appear in the From the Archives section, it will appear with probability proportional to the cube of its score.

Why These Features

LessWrong as a Repository for "long content"

Gwern’s about page has influenced me a lot in thinking about the future of LessWrong. Gwern uses the following quote:

The Internet is self destructing paper. A place where anything written is soon destroyed by rapacious competition and the only preservation is to forever copy writing from sheet to sheet faster than they can burn. If it’s worth writing, it’s worth keeping. If it can be kept, it might be worth writing…If you store your writing on a third party site like Blogger, Livejournal or even on your own site, but in the complex format used by blog/wiki software du jour you will lose it forever as soon as hypersonic wings of Internet labor flows direct people’s energies elsewhere. For most information published on the Internet, perhaps that is not a moment too soon, but how can the muse of originality soar when immolating transience brushes every feather?

-- Julian Assange (“Self destructing paper”, 5 December 2006)

Most of the content on the internet is designed to be read and forgotten in a very short period of time. Existing discussion platforms like Reddit and many forums even close threads automatically after a certain period of time to ensure that all discussion centers around the most recent activity on the site.

One of the goals I have with LessWrong is to be a place where we can build on each other's ideas over the course of multiple decades, and where if you show up, you engage with the content on the site in a focused way, more similar to a textbook than a normal internet forum. And like our best textbooks, good introductions into core topics tend to stand the test of time quite well (e.g. the Feynman Lectures, which is still one of, if not the best introduction to physics even 60 years later).

The Continue Reading system is a key part of that goal, because it makes it much easier to use the site as a tool for focused study, since continuing to read the sequences you started is now one of the core actions on the site.

The recommendation system is also a key part of that goal, because it creates a way to discover content from the complete history of LessWrong, instead of just the last week, which strikes me as a necessary component to make collective intellectual progress that can span multiple decades. The best things for almost anyone to read have very likely not been written in the past week.

LessWrong as a Nudge

Continue Reading is a nudge to encourage reading a few long things, rather than a lot of short things. Longer writing allows topics to be explored in greater depth, and also enables more explicit decisions about what to read, since making one decision per sequence is a lot less work than making one decision per post.

From the Archives is a nudge to read better posts. When we choose what to read, there is often a recency bias; the best writing of the past ten years will be better, on average, than the best writing of the past week, but active conversations will focus on the most recent things. A good information diet contains a mix of recent writing and of timeless classics; by putting From the Archives on the home page, we are saying, on the margin, read more of the past.

I also think that From the Archives will have a positive effect on what people write on LessWrong. There are many good ideas in LessWrong's archives, waiting to be built upon, which haven't received attention recently; my hope is that recommendations of older posts will inspire more good writing.

Caveat: Addictiveness

Reading the latest posts on LessWrong is finite; you will run out of interesting-seeming recent posts, which creates a natural limit on time spent. Reading posts from the archives is effectively infinite; LessWrong's archives are deep enough that you probably won't ever run out of things to read. These new recommendation features therefore offer an opportunity to spend a lot of time by accident. We'd rather you make a deliberate decision about what and how much to read on LessWrong.

If you find you're spending more time reading LessWrong's recommended posts than you want, or expect that you would spend more time than you want to, you can turn off the Continue Reading and/or From the Archives sections by clicking the gear icon. (This requires that you be logged in to save the setting.)

These Are Beta

These features are beta, and probably have bugs. The From the Archives post selection algorithm we're currently using (based on post scores) seems to work okay for now, but scores are heavily affected by post visibility as well as quality, so some posts (especially imported posts) aren't being recommended that should be, and post scores will suffer a positive-feedback effect where being recommended causes posts to be recommended more. So, we expect to rely less on the raw post score in the future, and more on other evaluation mechanisms such as asking users for retrospective evaluations of posts they've previously read, read-completion and clickthrough rates, vote-to-views ratios, etc. The recommendation algorithm is likely to become too complex to straightforwardly explain, though its workings will always be knowable to those willing to dive into the source code.

36 comments

Comments sorted by top scores.