Arbital scrape

post by emmab · 2019-06-06T23:11:17.943Z · score: 96 (31 votes) · LW · GW · 23 comments

Update: Arbital Scrape V2 [LW · GW]

I've scraped as the site is unusably slow and hard to search for me.

The scrape is locally browsable and plain HTML save for MathJax.

If there's interest let me know as I may tidy up and open source my code. edit: working on this



Comments sorted by top scores.

comment by Said Achmiz (SaidAchmiz) · 2019-06-07T00:35:27.344Z · score: 24 (10 votes) · LW · GW

This is now hosted at .

comment by Raemon · 2019-06-08T04:49:23.494Z · score: 5 (2 votes) · LW · GW

FWIW, I think it'd be helpful if this link were edited into the OP to make viewing more obvious.

comment by Said Achmiz (SaidAchmiz) · 2019-06-08T06:42:12.016Z · score: 3 (2 votes) · LW · GW

You may want to post a comment saying this as a top-level reply, so that emmab will be notified of it, yes?

comment by Raemon · 2019-06-08T06:47:54.123Z · score: 3 (1 votes) · LW · GW

Pretty sure he gets notified of all comments on his posts by default

comment by Said Achmiz (SaidAchmiz) · 2019-06-08T07:45:43.284Z · score: 3 (2 votes) · LW · GW

Wait… really? I thought the notifications were just for top-level comments?? Is that false…?

comment by Raemon · 2019-06-08T16:31:55.023Z · score: 3 (1 votes) · LW · GW

I’m not actually sure and can’t check right now, but I thought so.

comment by habryka (habryka4) · 2019-06-08T17:17:02.486Z · score: 4 (2 votes) · LW · GW

Yeah, notifications are for all comments, not just top-level.

comment by emmab · 2019-06-08T18:33:32.041Z · score: 1 (1 votes) · LW · GW


comment by Raemon · 2019-06-07T05:35:58.541Z · score: 11 (2 votes) · LW · GW


comment by Raemon · 2019-06-07T05:40:49.438Z · score: 11 (2 votes) · LW · GW

Follow up: wow, there's a lot of good stuff here I hadn't known about.

comment by clone of saturn · 2019-06-07T04:23:01.514Z · score: 10 (3 votes) · LW · GW

I've been working on something similar, it should be ready soon-ish.

comment by emmab · 2019-06-07T07:47:23.126Z · score: 1 (1 votes) · LW · GW

I'll wait to open source mine to see if yours is better then :)

comment by emmab · 2019-06-25T10:06:58.733Z · score: 3 (2 votes) · LW · GW

Nvm. Arbital Scrape V2 [LW · GW]

comment by riceissa · 2019-06-07T03:20:28.092Z · score: 10 (6 votes) · LW · GW

The scrape seems to be missing the Solomonoff induction dialogue, which was available at (at the moment I just get an error).

ETA (2019-06-28): The new version of the scrape has this page, and can be viewed on the GreaterWrong version.

comment by roland · 2019-06-07T19:25:08.013Z · score: 7 (4 votes) · LW · GW

Seconded, that part is missing. Thanks for pointing out that very interesting dialogue.

comment by Smaug123 · 2019-06-07T06:15:45.540Z · score: 8 (6 votes) · LW · GW

Thanks very much for this! I've written a lot of stuff on there (I'm the Patrick Stevens whose name is splatted all over the screenshot). I asked them a year ago (ish) whether I could have a data dump, and they said it was Too Difficult; and I didn't bother scraping it myself. I'm glad you actually went and did something about it!

comment by Said Achmiz (SaidAchmiz) · 2019-06-07T00:24:46.865Z · score: 6 (3 votes) · LW · GW

If there’s interest let me know as I may tidy up and open source my code.

Please do!

comment by Liron · 2019-06-07T01:55:15.424Z · score: 5 (3 votes) · LW · GW

Nice project thanks. It's also consistent with my analysis that Arbital's fancy wiki features never added much value (say, more than +30%) beyond the value that that content would have if published in a pre-existing format and venue.

comment by habryka (habryka4) · 2019-06-07T00:03:48.472Z · score: 5 (3 votes) · LW · GW

This seems good. I remember Said being interested in hosting a static version of the site.

comment by roland · 2019-06-20T12:19:17.262Z · score: 3 (2 votes) · LW · GW

Just a reminder, the Solomonoff induction dialogue is still missing: [LW · GW]

comment by emmab · 2019-06-25T10:03:48.296Z · score: 2 (2 votes) · LW · GW

See Arbital Scrape V2 [LW · GW]

comment by digital_carver · 2019-07-05T07:23:27.342Z · score: 1 (1 votes) · LW · GW

For the sake of us newbies, could you explain what the site was/is?

comment by Raemon · 2019-07-05T20:11:30.861Z · score: 5 (2 votes) · LW · GW

Arbital Postmortem is the first explanation I can think of.