Is there a lesswrong archive of all public posts?
post by Nicolas Lacombe (nicolas-lacombe) · 2023-11-08T19:26:45.030Z · LW · GW · 7 commentsThis is a question post.
Contents
7 comments
like for example: the wikimedia db dumps, or the stack exchange db dumps.
i'd like to be able to browse lesswrong while offline. even better if i can process the data with a script while offline.
it would also be useful for backup purposes: if ever something happens to the site in the longterm, some of it's users might have an exact copy that can be corss-compared and used to restore/share its content.
related:
- Why is lesswrong blocking wget and curl (scrape)? [LW · GW]
- Can I archive content from lesswrong.com on the wayback machine (internet archive, archive.org) ? [LW · GW]
Answers
7 comments
Comments sorted by top scores.
comment by Gunnar_Zarncke · 2023-11-09T07:00:55.939Z · LW(p) · GW(p)
There used to be a downloadable archive of all posts on the old LW 1.0 site. I think there was a SiteMeta post about it, but I can't find it, but anyway that link wouldn't work anymore. I think such an archive it's a nice thing to have and also avoids unnecessary scraping.
comment by Nicolas Lacombe (nicolas-lacombe) · 2023-11-08T20:11:02.560Z · LW(p) · GW(p)
@gwern [LW · GW] : is this something you'd be interested in having?
Replies from: gwern↑ comment by gwern · 2023-11-11T01:43:35.753Z · LW(p) · GW(p)
Not particularly now that GW exists. Its posts are easy to archive, and if I need something, I can always just ask saturn2.
Replies from: nicolas-lacombe, nicolas-lacombe↑ comment by Nicolas Lacombe (nicolas-lacombe) · 2023-11-11T02:13:09.967Z · LW(p) · GW(p)
thanks for answering. who/what is "saturn2" ?
Replies from: mesaoptimizer↑ comment by mesaoptimizer · 2023-11-11T13:26:41.973Z · LW(p) · GW(p)
https://www.lesswrong.com/posts/66DXhQJyPEJNsXgfw/an-alternative-way-to-browse-greaterwrong-2-0 [LW · GW]
@clone of saturn [LW · GW], aka "saturn2", runs GW
↑ comment by Nicolas Lacombe (nicolas-lacombe) · 2023-11-11T02:23:14.151Z · LW(p) · GW(p)
does greaterwrong.com have archive files that i can download? or do it need to archive it myself (ex: wget -mk
) ?
↑ comment by mesaoptimizer · 2023-11-11T13:27:22.742Z · LW(p) · GW(p)
You should probably ask saturn2 for permission before doing so.