Posts

We've built Connected Papers - a visual tool for researchers to find and explore academic papers 2020-06-08T18:44:56.467Z

Comments

Comment by discordy on We've built Connected Papers - a visual tool for researchers to find and explore academic papers · 2022-03-23T21:24:44.691Z · LW · GW

Hi Jane, feel free to write to us at hello@connectedpapers.com with more details. We're very short on time so not sure we can partner on additional projects right now.

Comment by discordy on We've built Connected Papers - a visual tool for researchers to find and explore academic papers · 2022-03-23T21:23:30.419Z · LW · GW

Hi Algon, thank you for the feedback.
I'm not sure we have easy access to this metadata about the authors. Even if we did, this feature would probably not be very high on our priority list so I wouldn't expect it soon. Sorry!

Comment by discordy on We've built Connected Papers - a visual tool for researchers to find and explore academic papers · 2022-03-23T21:18:11.099Z · LW · GW

Hey, Said. That's some great feedback! I'll share it with the team.
We've been debating for some time about the best way to design this interaction (open a node as a graph). Your suggestions are great and we'll definitely experiment with them. Thank you!

Comment by discordy on We've built Connected Papers - a visual tool for researchers to find and explore academic papers · 2020-06-11T09:32:55.559Z · LW · GW

Got it, thanks for the suggestions - we'll definitely brainstorm about this.

In the meantime, mouse-over shows an immediate preview of the title on the right side panel - hopefully that helps you use the graph.

Comment by discordy on We've built Connected Papers - a visual tool for researchers to find and explore academic papers · 2020-06-11T09:30:40.594Z · LW · GW

Yes - these are probably our most requested features and are high in our list of features to add.

Comment by discordy on We've built Connected Papers - a visual tool for researchers to find and explore academic papers · 2020-06-10T13:23:15.579Z · LW · GW

Really happy to hear that Adam!

Comment by discordy on We've built Connected Papers - a visual tool for researchers to find and explore academic papers · 2020-06-10T13:21:35.426Z · LW · GW

Hey, thanks for the feedback. We're aware and there's much more work to be done. Added your comment to our list of things to do.

Comment by discordy on We've built Connected Papers - a visual tool for researchers to find and explore academic papers · 2020-06-10T13:20:06.937Z · LW · GW

The problem with doing that is that paper titles are extremely long and clutter the graph too much, and if we only show a few words many nodes get the same title.

As an alternative suggestion: would showing the titles upon instant mouse-over satisfy your need?

Comment by discordy on We've built Connected Papers - a visual tool for researchers to find and explore academic papers · 2020-06-10T13:15:54.575Z · LW · GW

Hey, glad you like the concept!

Sorry for the malfunction you experienced, it probably happened while we were overloaded. We've since increased the server count and limited the amount of graphs users can build in parallel.

An insider tip: you only have waiting times for graphs that have never been built before. If you return to the graph you've already built, it would be instantaneous.

Comment by discordy on We've built Connected Papers - a visual tool for researchers to find and explore academic papers · 2020-06-10T13:14:36.797Z · LW · GW

Hey, glad to see you like the concept! We're actively working on improving the performance.

1. Everything is proprietary for now. After consideration we decided that this project is not well suited for open sourcing at this time.

2. Graphs are generated on the fly, but only for the first time. We keep the results in a cache so when another user asks for the same graph later, they'd get it instantly. Also, asking for graphs which are close in paper-space would also run faster.

3. We rely on external sources (like the Open Corpus by Semantic Scholar) for the citations database. Unfortunately, no database is perfect yet and sometimes citations are badly parsed.

4. First, we found this tool very fun for exploring paper-space in new domains. I sometimes just enter a keyword like "psychology" and start exploring. This gives me a nice overview of the type of titles and branches in new (for me) fields of science.

Second, I was surprised with how easy it was to recognize papers that are bridging multiple disciplines. Take a look at our example graph "deepfruits", for example: there are two obvious clusters. One shows deep learning papers mostly about detection. The other shows papers that describe how these techniques were applied in agriculture.

5. We've experimented early on and arrived to a conclusion that more than ~50 papers on the screen is too much clutter, and it's better to traverse paper-space by building more graphs. Avoiding specifics on purpose :)

Comment by discordy on We've built Connected Papers - a visual tool for researchers to find and explore academic papers · 2020-06-10T12:35:23.519Z · LW · GW

Answered above :)

Comment by discordy on We've built Connected Papers - a visual tool for researchers to find and explore academic papers · 2020-06-10T12:34:00.425Z · LW · GW

Hey, thanks for the compliment!

Sorry for the malfunction you experienced, it probably happened while we were overloaded. We've since increased the server count and limited the amount of graphs users can build in parallel.

An insider tip: you only have waiting times for graphs that have never been built before. If you return to the graph you've already built, it would be instantaneous.

Comment by discordy on We've built Connected Papers - a visual tool for researchers to find and explore academic papers · 2020-06-10T09:35:16.525Z · LW · GW

Hi!

Sorry for the delayed reply - this launch has gone far better than we anticipated and our servers and infrastructure have been overwhelmed, even after increasing our auto-scaling ceiling twice... So the last day was a bit hectic :)

This is very relevant to your question.

Connected Papers was built with altruistic intentions, and indeed we expected to fund the servers ourselves - this would be our effective altruism cause. But, it looks like with the demand we're seeing our budgets would not be enough.

We're now brainstorming various solutions, including pro plans for the product (we're committed that there will always be a free version that's at least as good as what we're providing now) and sponsorship from cloud providers. For the time being, we've added a donation button which will go directly to funding the servers - active donations would be a good indicator that some users are willing to pay for the service.

---

The other (much smaller at the moment) risk for Connected Papers is that it does not own the data on which the graphs are built. We are relying on external datasets like the Open Corpus by Semantic Scholar. This means that if at some point they change their license it could impact our ability to provide service or the quality of our graphs. This does not seem like a big risk at the moment as we're actively engaged with Semantic Scholar and have their blessing (e.g. they tweeted about us).

In addition, any errors in the datasets reflect poorly in our graphs. In fact, it appears that our graphs are a very good method to detect bad entries in the DB. Ideally we would want to allow users to report bad entries and close the loop with Semantic Scholar at the database level.


tl;dr

We need to solve server funding, we rely on external sources for the data on which graphs are built.