Arbital scrape

post by emmab · 2019-06-06T23:11:17.943Z · score: 96 (31 votes) · LW · GW · 23 comments

Update: Arbital Scrape V2 [LW · GW]

I've scraped http://arbital.com as the site is unusably slow and hard to search for me.

The scrape is locally browsable and plain HTML save for MathJax.

https://drive.google.com/open?id=1b7dKhOzfMpFwngAeI8efeOzv147Lv_mx

https://emma-borhanian.github.io/arbital-scrape/

If there's interest let me know as I may tidy up and open source my code. edit: working on this

Mirror: www.obormot.net/arbital

23 comments

Comments sorted by top scores.

comment by Said Achmiz (SaidAchmiz) · 2019-06-07T00:35:27.344Z · score: 24 (10 votes) · LW · GW

This is now hosted at https://www.obormot.net/arbital/ .

comment by Raemon · 2019-06-08T04:49:23.494Z · score: 5 (2 votes) · LW · GW

FWIW, I think it'd be helpful if this link were edited into the OP to make viewing more obvious.

comment by Said Achmiz (SaidAchmiz) · 2019-06-08T06:42:12.016Z · score: 3 (2 votes) · LW · GW

You may want to post a comment saying this as a top-level reply, so that emmab will be notified of it, yes?

comment by Raemon · 2019-06-08T06:47:54.123Z · score: 3 (1 votes) · LW · GW

Pretty sure he gets notified of all comments on his posts by default

comment by Said Achmiz (SaidAchmiz) · 2019-06-08T07:45:43.284Z · score: 3 (2 votes) · LW · GW

Wait… really? I thought the notifications were just for top-level comments?? Is that false…?

comment by Raemon · 2019-06-08T16:31:55.023Z · score: 3 (1 votes) · LW · GW

I’m not actually sure and can’t check right now, but I thought so.

comment by habryka (habryka4) · 2019-06-08T17:17:02.486Z · score: 4 (2 votes) · LW · GW

Yeah, notifications are for all comments, not just top-level.

comment by emmab · 2019-06-08T18:33:32.041Z · score: 1 (1 votes) · LW · GW

Ok

comment by Raemon · 2019-06-07T05:35:58.541Z · score: 11 (2 votes) · LW · GW

Thanks!

comment by Raemon · 2019-06-07T05:40:49.438Z · score: 11 (2 votes) · LW · GW

Follow up: wow, there's a lot of good stuff here I hadn't known about.

comment by clone of saturn · 2019-06-07T04:23:01.514Z · score: 10 (3 votes) · LW · GW

I've been working on something similar, it should be ready soon-ish.

comment by emmab · 2019-06-07T07:47:23.126Z · score: 1 (1 votes) · LW · GW

I'll wait to open source mine to see if yours is better then :)

comment by emmab · 2019-06-25T10:06:58.733Z · score: 3 (2 votes) · LW · GW

Nvm. Arbital Scrape V2 [LW · GW]

comment by riceissa · 2019-06-07T03:20:28.092Z · score: 10 (6 votes) · LW · GW

The scrape seems to be missing the Solomonoff induction dialogue, which was available at https://arbital.com/p/solomonoff_induction/?l=1hh (at the moment I just get an error).

ETA (2019-06-28): The new version of the scrape has this page, and can be viewed on the GreaterWrong version.

comment by roland · 2019-06-07T19:25:08.013Z · score: 7 (4 votes) · LW · GW

Seconded, that part is missing. Thanks for pointing out that very interesting dialogue.

comment by Smaug123 · 2019-06-07T06:15:45.540Z · score: 8 (6 votes) · LW · GW

Thanks very much for this! I've written a lot of stuff on there (I'm the Patrick Stevens whose name is splatted all over the screenshot). I asked them a year ago (ish) whether I could have a data dump, and they said it was Too Difficult; and I didn't bother scraping it myself. I'm glad you actually went and did something about it!

comment by Said Achmiz (SaidAchmiz) · 2019-06-07T00:24:46.865Z · score: 6 (3 votes) · LW · GW

If there’s interest let me know as I may tidy up and open source my code.

Please do!

comment by Liron · 2019-06-07T01:55:15.424Z · score: 5 (3 votes) · LW · GW

Nice project thanks. It's also consistent with my analysis that Arbital's fancy wiki features never added much value (say, more than +30%) beyond the value that that content would have if published in a pre-existing format and venue.

comment by habryka (habryka4) · 2019-06-07T00:03:48.472Z · score: 5 (3 votes) · LW · GW

This seems good. I remember Said being interested in hosting a static version of the site.

comment by roland · 2019-06-20T12:19:17.262Z · score: 3 (2 votes) · LW · GW

Just a reminder, the Solomonoff induction dialogue is still missing:

https://www.lesswrong.com/posts/muKEBrHhETwN6vp8J/arbital-scrape#tKgeneD2ZFZZxskEv [LW · GW]

comment by emmab · 2019-06-25T10:03:48.296Z · score: 2 (2 votes) · LW · GW

See Arbital Scrape V2 [LW · GW]

comment by digital_carver · 2019-07-05T07:23:27.342Z · score: 1 (1 votes) · LW · GW

For the sake of us newbies, could you explain what the site was/is?

comment by Raemon · 2019-07-05T20:11:30.861Z · score: 5 (2 votes) · LW · GW

Arbital Postmortem is the first explanation I can think of.