All discussion post titles, points, and dates as an excel sheet
post by Alexandros · 2014-06-03T14:38:20.341Z · LW · GW · Legacy · 10 commentsYou can find it here.
Earlier today I wanted to quantify whether lesswrong has stopped being a well kept garden. So I wrote a scraper to produce the above dataset, so that anyone that wants to do the analysis, can.
All data is as of a few minutes ago.
For programmers: You can see the source here, it's made to run on scraperwiki, but it will time out after about 3000 articles. At that point you need to adjust the initial value of the uri variable to be the last uri printed. Repeating this process once more will allow you to reach the end. Have fun.
10 comments
Comments sorted by top scores.
comment by Lumifer · 2014-06-03T17:24:11.516Z · LW(p) · GW(p)
Earlier today I wanted to quantify whether lesswrong has stopped being a well kept garden.
Then you probably should start by quantifying what does "being a well kept garden" mean.
Replies from: Alexandros↑ comment by Alexandros · 2014-06-03T18:35:07.952Z · LW(p) · GW(p)
True. I guess I was being a bit cheeky. LW is no longer being kept at all AFAICT (or just on maintenance), just wanted to see if it's on an upward or downward trajectory. I obviously think there is a problem, and I have a solution to suggest, but I wanted to double check my intuition with the numbers.
comment by Error · 2014-06-03T17:07:55.421Z · LW(p) · GW(p)
Authors might be an interesting field to add; one of the more plausible measures mentioned in the other thread was a drop in posts from specific prolific authors.
Replies from: Alexandros↑ comment by Alexandros · 2014-06-03T18:33:32.945Z · LW(p) · GW(p)
post updated with code, go crazy! number of comments is another one I'd add if I ran it again.
comment by Richard_Kennaway · 2014-06-03T19:56:50.819Z · LW(p) · GW(p)
Earlier today I wanted to quantify whether lesswrong has stopped being a well kept garden.
Before you look at the numbers, what metrics are you going to use to quantify this?
Replies from: Alexandros↑ comment by Alexandros · 2014-06-03T20:41:34.341Z · LW(p) · GW(p)
posts per month, upvotes per month. (i understand score is positive minus negative, but it cancels out). potentially comments per month too, but I didn't fetch that data. substitute month for your preferred granularity of course.
comment by Dr_Manhattan · 2014-06-03T16:22:01.197Z · LW(p) · GW(p)
for +10 points, post the scraper. (but put a throttle in by default)
Replies from: Alexandros↑ comment by Alexandros · 2014-06-03T18:32:59.625Z · LW(p) · GW(p)
done
comment by Gunnar_Zarncke · 2014-06-03T16:11:55.909Z · LW(p) · GW(p)
I wanted to quantify whether lesswrong has stopped being a well kept garden.
I'm very curious about you results.
Replies from: Alexandros↑ comment by Alexandros · 2014-06-03T18:24:54.838Z · LW(p) · GW(p)
Well, it's not being 'kept' anymore for one, but I didn't need analysis for that. I guess the question is if it is flourishing or dying out.