Arbital Scrape V2
post by emmab · 2019-06-25T10:03:21.962Z · LW · GW · 6 commentsContents
6 comments
I've scraped http://arbital.com as the site is unusably slow and hard to search for me.
https://drive.google.com/open?id=1b7dKhOzfMpFwngAeI8efeOzv147Lv_mx
https://mega.nz/#!SUQg1YxS!G7bmpNipcl1Ztugvr6l6sUJUu1D0gDg1-jA6oXLNEWg
https://emma-borhanian.github.io/arbital-scrape/
The scrape is locally browsable and plain HTML save for MathJax and a few interactive demos. Source code included (with git history).
(previously Arbital Scrape [LW · GW])
Updates: Included source code, MathJax and link formatting, cross-linking, missing pages, etc
Source: https://github.com/emma-borhanian/arbital-scrape
Mirror: www.obormot.net/arbital
6 comments
Comments sorted by top scores.
comment by Said Achmiz (SaidAchmiz) · 2019-06-25T17:07:20.287Z · LW(p) · GW(p)
I’ve updated www.obormot.net/arbital with the new scrape.
The math indeed works now; see this page for an example.
Replies from: emmabcomment by emmab · 2019-06-25T10:47:36.512Z · LW(p) · GW(p)
Please do not re-download the pages from arbital.com without good reason. I've added a single line of code to disable this. This is why I'm not uploading the source code to github, but did include it in the zip file you can download.
Running the code as-is will simply regenerate the HTML using the already-downloaded raw json.
Edit: This is being downvoted. I'm happy to reevaluate this and upload to github instead of merely including the source in the zip file. Please comment if this is what you wish.
Replies from: SaidAchmiz, emmab↑ comment by Said Achmiz (SaidAchmiz) · 2019-06-25T17:09:59.345Z · LW(p) · GW(p)
Regardless of whether you upload to GitHub, would you mind adding a license for the source code?
Replies from: emmab↑ comment by emmab · 2019-06-26T01:55:41.050Z · LW(p) · GW(p)
Decided to upload source to github now that I know arbital's license: https://github.com/emma-borhanian/arbital-scrape