Listen to top LessWrong posts with The Nonlinear Library

post by KatWoods (ea247) · 2021-10-19T12:24:43.564Z · LW · GW · 27 comments

Contents

      Or, just search for it in your preferred podcasting app.
  Goal: increase the number of people who read EA research
  Why it’s useful
    EA needs more audio content
    There are a lot of listeners
    Existing text-to-speech solutions are sub-optimal
    How The Nonlinear Library fixes these problems
    Why not have a human read the content?
  Future Playlists (“Bookshelves”)
  Who we are
  Footnotes
None
28 comments

Crossposted from the EA Forum.

We are excited to announce the launch of The Nonlinear Library, which allows you to easily listen to top EA and rationalist content on your podcast player. We use text-to-speech software to create an automatically updating repository of audio content from the EA Forum, Alignment Forum, LessWrong, and other EA blogs.

In the rest of this post, we’ll explain our reasoning for the audio library, why it’s useful, why it’s potentially high impact, its limitations, and our plans. You can read it here or listen to the post in podcast form here.

Listen here: Spotify, Google Podcasts, Pocket Casts, Apple, or elsewhere

Or, just search for it in your preferred podcasting app.

Goal: increase the number of people who read EA research

A koan: if your research is high quality, but nobody reads it, does it have an impact?

Generally speaking, the theory of change of research is that you investigate an area, come to better conclusions, people read those conclusions, they make better decisions, all ultimately leading to a better world. So the answer is no. Barring some edge cases (1), if nobody reads your research, you usually won’t have any impact.

Research → Better conclusion → People learn about conclusion → People make better decisions → The world is better

Nonlinear is working on the third step of this pipeline: increasing the number of people engaging with the research. By increasing the total number of EA and rationalist articles read, we’re increasing the impact of all of that content.

This is often relatively neglected because researchers typically prefer doing more research instead of promoting their existing output. Some EAs seem to think that if their article was promoted one time, in one location, such as the EA Forum, then surely most of the community saw it and read it. In reality, it is rare that more than a small percentage of the community will read even the top posts. This is an expected-value tragedy, when a researcher puts hundreds of hours of work into an important report which only a handful of people read, dramatically reducing its potential impact.

Here are some purely hypothetical numbers just to illustrate this way of thinking:

Imagine that you, a researcher, have spent 100 hours producing outstanding research that is relevant to 1,000 out of a total of 10,000 EAs.

Each relevant EA who reads your research will generate $1,000 of positive impact. So, if all 1,000 relevant EAs read your research, you will generate $1 million of impact.

You post it to the EA Forum, where posts receive 500 views on average [EA · GW]. Let’s say, because your report is long, only 20% read the whole thing - that’s 100 readers. So you’ve created 100*1,000 = $100,000 of impact. Since you spent 100 hours and created $100,000 of impact, that’s $1,000 per hour - pretty good!

But if you were to spend, say 1 hour, promoting your report -  for example, by posting links on EA-related Facebook groups - to generate another 100 readers, that would produce another $100,000 of impact. That’s $100,000 per marginal hour or ~$2,000 per hour taking into account the fixed cost of doing the original research.

Likewise, if another 100 EAs were to listen to your report while commuting, that would generate an incremental $100,000 of impact - at virtually no cost, since it’s fully automated.

In this illustrative example, you’ve nearly tripled your cost-effectiveness and impact with one extra hour spent sharing your findings and having a public system that turns it into audio for you.  

Another way the audio library is high expected value is that instead of acting as a multiplier on just one researcher or one organization, it acts as a multiplier on nearly the entire output of the EA research community. This allows for two benefits: long-tail capture and the power of large numbers and multipliers.

Long-tail capture. The value of research is extremely long tailed, with a small fraction of the research having far more impact than others. Unfortunately, it’s not easy to do highly impactful research or predict in advance which topics will lead to the most traction. If you as a researcher want to do research that dramatically changes the landscape, your odds are low. However, if you increase the impact of most of the EA community’s research output, you also “capture” the impact of the long tails when they occur. Your probability of applying a multiplier to very impactful research is actually quite high.

Power of large numbers and multipliers. If you apply a multiplier to a bigger number, you have a proportionately larger impact. This means that even a small increase in the multiplier leads to outsized improvements in output. For example, if a single researcher toiled away to increase their readership by 50%, that would likely have a smaller impact than the Nonlinear Library increasing the readership of the EA Forum by even 1%. This is because 50% times a small number is still very small, whereas 1% times a large number is actually quite large. And there’s reason to believe that the library could have much larger effects on readership, which brings us to our next section. 


Why it’s useful


EA needs more audio content

EA has a vibrant online community, and there is an amazing amount of well researched, insightful, and high impact content. Unfortunately, it’s almost entirely in writing and very little is in audio format.

There are a handful of great podcasts, such as the 80,000 Hours and FLI podcasts, and some books are available on Audible. However, these episodes come out relatively infrequently and the books even less so. There’s a few other EA-related podcasts [EA · GW], including one for the EA Forum, but a substantial percentage have become dormant, as is far too common for channels because of the considerable amount of effort required to put out episodes.

There are a lot of listeners

The limited availability of audio is a shame because many people love to listen to content. For example, ever since the 80,000 Hours podcast came out, a common way for people to become more fully engaged in EA is to mainline all of their episodes. Many others got involved through binging the HPMOR audiobook, as Nick Lowry puts it in this meme. We are definitely a community of podcast listeners.

Why audio? Often, you can’t read with your eyes but you can with your ears. For example, when you’re working out, commuting, or doing chores. Sometimes it’s just for a change of pace. In addition, some people find listening to be easier than reading. Because it feels easier, they choose to spend time learning that might otherwise be spent on lower value things.

Regardless, if you like to listen to EA content, you’ll quickly run out of relevant podcasts - especially if you’re listening at 2-3x speed - and have to either use your own text-to-speech software or listen to topics that are less relevant to your interests.
 

Existing text-to-speech solutions are sub-optimal

We’ve experimented extensively with text-to-speech software over the years, and all of the dozens of programs we’ve tried have fairly substantial flaws. In fact, a huge inspiration for this project was our frustration with the existing solutions and thinking that there must be a better way. Here are some of the problems that often occur with these apps:

In the end, this leads to only the most motivated people using the services, leaving out a huge percentage of the potential audience. (2)

How The Nonlinear Library fixes these problems

To make it as seamless as possible for EAs to use, we decided to release it as a podcast so you can use the podcast app you’re already familiar with. Additionally, podcast players tend to be reasonably well designed and offer great customizability of playlists and speeds.

We’re paying for some of the best AI voices because old voices suck. And we spent a bunch of time fixing weird formatting errors and mispronunciations and have a system to fix other recurring ones. If you spot any frequent mispronunciations or bugs, please report them in this form so we can continue improving the service.

Initially, as an MVP, we’re just posting each day’s top upvoted articles from the EA Forum, Alignment Forum, and LessWrong. (3) We are planning on increasing the size and quality of the library over time to make it a more thorough and helpful resource.

Why not have a human read the content?

The Astral Codex Ten podcast and other rationalist podcasts do this. We seriously considered this, but it’s just too time consuming, and there is a lot of written content. Given the value of EA time, both financially and counterfactually, this wasn’t a very appealing solution. We looked into hiring remote workers but that would still have ended up costing at least $30 an episode. This compared to approximately $1 an episode via text-to-speech software.

On top of the time costs leading to higher monetary costs, it also makes us able to make a far more complete library. If we did this with humans and we invested a ton of time and management, we might be able to convert seven articles a week. At that rate, we’d never be able to keep up with new posts, let alone include the historical posts that are so valuable. With text-to-speech software, we could have the possibility of keeping up with all new posts and converting the old ones, creating a much more complete repository of EA content. Just imagine being able to listen to over 80% of EA writing you’re interested in compared to less than 1%.

Additionally, the automaticity of text-to-speech fits with Nonlinear’s general strategy of looking for interventions that have “passive impact”. Passive impact is the altruistic equivalent of passive income, where you make an upfront investment and then generate income with little to no ongoing maintenance costs. If we used human readers, we’d have a constant ongoing cost of managing them and hiring replacements. With TTS, after setting it up, we can mostly let it run on its own, freeing up our time to do other high impact activities.

Finally, and least importantly, there is something delightfully ironic about having an AI talk to you about how to align future AI.

On a side note, if for whatever reason you would not like your content in The Nonlinear Library, just fill out this form. We can remove that particular article or add you to a list to never add your content to the library, whichever you prefer. 


Future Playlists (“Bookshelves”)

There are a lot of sub-projects that we are considering doing or are currently working on. Here are some examples:

Who we are

We're Nonlinear, a meta longtermist organization focused on reducing existential and suffering risks. More about us.

Footnotes

(1)  Sometimes the researcher is the same person as the person who puts the results into action, such as Charity Entrepreneurship’s model. Sometimes it’s a longer causal chain, where the research improves the conclusions of another researcher, which improves the conclusions of another researcher, and so forth, but eventually it ends in real world actions. Finally, there is often the intrinsic happiness of doing good research felt by the researcher themselves.


(2)  For those of you who want to use TTS for a wider variety of articles than what the Nonlinear Library will cover, the ones I use are listed below. Do bear in mind they each have at least one of the cons listed above. There are probably also better ones out there as the landscape is constantly changing. 

(3) The current upvote thresholds for which articles are converted are:
25 for the EA forum
30 for LessWrong
No threshold for the Alignment Forum due to low volume

This is based on the frequency of posts, relevance to EA, and quality at certain upvote levels.
 

27 comments

Comments sorted by top scores.

comment by Yoav Ravid · 2021-10-19T17:47:49.963Z · LW(p) · GW(p)

This is great! I have a dyslectic friend that may benefit from this, so I'll be sure to tell him.

I think it would be great to go even further and have this as an in-built feature on the website, so (ideally) for any every article you could click on a listen to this article button and listen to an auto-generated reading. This has far less friction and is more accessible to those who don't know about the library or don't use any podcast apps (like me). It can also scale better for multiple voices, if that's a desired feature. 

ETA: If it's an in-built feature then it can also be applied to comments and tags, which could also be very useful.

If creating an audio version of each post is too expensive, then perhaps it could be limited to a certain Karma score like you're doing right now, or only do it for posts when someone first clicks the button.

Perhaps as a step towards that something can be added to the post page on the site that shows when there's an audio version available and either links to Spotify or directly to the MP3 file?

I would like to see older posts also get audio versions, there are many good old posts and I don't see a reason to heavily prioritize new posts (also, more selfishly, I would love to have audio versions of my own posts that pass the threshold but were posted before this project began)

Question: If an article passes the karma threshold only a week after it's posted, will it still be narrated? In other words, what's the threshold to pass the threshold? :)

(Two small suggestions: I'd put the link to the audio version of this post before the first paragraph, and I'd add another link to the library at the end of the post.)

Replies from: Yoav Ravid
comment by Yoav Ravid · 2021-10-27T18:23:28.464Z · LW(p) · GW(p)

Some feedback after trying it for a bit. 

First of all - the narration quality is amazing. I had no idea such good narration exists. I listened to all [EA · GW] the [EA · GW] people [EA · GW] you [EA · GW] could [EA · GW] come [EA · GW] to [EA · GW] love [EA · GW] while also reading it, and the narration only flopped when there was some sort of typo in the text itself (like forgetting a space after the dot at the end of a sentence, "like.so", making it read out the dot). There were several sentences that a human could have done much better due to understanding the intentions and being able to match the intonation to fit, but it didn't feel deal-breaking (even though it was fiction, which I expect is harder to narrate well).

With that, it feels pretty inaccessible to me. In large part it's because I don't use any podcast app, (also uploading it to YouTube would help), but even if I did, then when I want to listen to a post instead of reading it, I have to find the nonlinear library page on one of those apps, then search for the specific posts there (hopefully not by scrolling, but from my experience this seems to be the only way :/). I guess this isn't a problem for those who use it instead of the frontpage of LW/EAF, which are the main audience for this, but it would still be nice if it worked better for "dual users". Hopefully in the future my suggestion of integrating it to the website would be implemented, cause I think that's the best way to do it.

comment by Ruby · 2021-10-19T16:56:05.510Z · LW(p) · GW(p)

My current policy* is to frontpage feature announcements since they are of broad interest and are kind of timeless content (I believe this was not always LW mod policy).

This service, though not from the LW team, seems tantamount to a feature announcement, so I'm frontpaging it too.

*The frontpage/personal distinction and associated policy is a bit confusing. It's on my to-do list to clean that up.

comment by Steven Byrnes (steve2152) · 2021-10-20T23:19:29.365Z · LW(p) · GW(p)

I like it! Listened to a couple things while doing laundry.

Minor suggestion FWIW: Make headings stand out more, somehow. Not sure how. Say the word "heading"? Switch to a different voice? Pause longer before the heading? I don't know, just wondering if anything is easy to do.

Replies from: EmersonSpartz
comment by Emerson Spartz (EmersonSpartz) · 2021-10-21T07:37:15.290Z · LW(p) · GW(p)

Great idea! We'll add it to the list.

comment by Alex_Altair · 2021-10-23T00:09:13.790Z · LW(p) · GW(p)

I'm psyched for this!

I've listened to a couple posts already, and my main specific suggestion is to have some sort of tone or jingle between the podcast intro and post, and between the end of post and outro. That would more clearly indicate the transition, and would at least slight breakup the voice monotony.

The only other things I noticed are small issues with the TTS, which I assume you can't control. (Examples include reading "..." as the word "dots", and pausing after an initial as if it were the end of a sentence.)

I think there's a good chance that this significantly upticks my reading of these forums.

Replies from: ea247
comment by KatWoods (ea247) · 2021-10-23T17:01:04.149Z · LW(p) · GW(p)

I'm so glad that you're enjoying it and finding it useful! 

Re:jingle, good idea. Since so many of the episodes are short we were trying to keep the intros and outros as minimal as possible, but I do think some sort of audio indicator, especially for the outro, would be helpful. 

comment by Gunnar_Zarncke · 2021-10-19T17:26:15.625Z · LW(p) · GW(p)

Sounds like a great product. You could offer this as a service for other forums, libraries etc. for profit.

Replies from: Yoav Ravid
comment by Yoav Ravid · 2021-10-19T17:35:16.265Z · LW(p) · GW(p)

And use the profit to narrate even more LessWrong/EA posts! Double win :)

comment by Alex_Altair · 2022-01-07T00:48:42.671Z · LW(p) · GW(p)

I've been subscribed to the podcast now for a couple months. Overall, I quite like it, but there's one main problem; the volume is way too high. When I go to listen to podcasts, I go to the most recently released episodes. I subscribe to a few dozen podcasts, and the Nonlinear Library episodes tend to drown out everything else, to the point where yesterday I decided to unsubscribe.

Some ways this might be fixed;

  • Raising the karma threshold for what gets converted into a podcast
  • Having a length threshold (is it worth showing a 5 minute read? I dunno, maybe)
  • Having multiple podcasts. Maybe one can be "all", maybe one can be "top posts", maybe they can be separated by tags, etc.

Overall I think there might be a conflict between the goal of "Make a LW podcast" and "make all of LW available via audio". Podcasts tend to be a certain length, a certain type of content, et cetera, and many LW posts don't quite make sense to include in that.

Replies from: Raemon
comment by Raemon · 2022-01-07T01:32:50.739Z · LW(p) · GW(p)

Note I initially interpreted this to mean "the audio volume is too high", although I figured it out by the end.

comment by Nihal M (nihal-m) · 2021-10-29T07:20:37.898Z · LW(p) · GW(p)

Thanks for doing this. I enjoy listening to audio form of content, and this makes reading my backlog of lesswrong and EA content very accessible. 

Minor suggestion. - The podcast is hosted on multiple platforms, but is missing YouTube. There are several podcasts that are audio only on Youtube( Naval Ravikant's podcast, Cortex by CGP Gray, etc.). Adding it to you tube can increase reach to potential new listeners. 

Replies from: ea247
comment by KatWoods (ea247) · 2021-10-30T21:49:23.031Z · LW(p) · GW(p)

100% agreed! Definitely on the list of things to do. 

comment by Gordon Seidoh Worley (gworley) · 2021-10-19T23:46:55.614Z · LW(p) · GW(p)

I don't listen to podcasts, so this doesn't totally fit for me, and I like to listen to stuff not on LW. So, in the interest of just sharing adjacent alternatives in case this idea sounds interesting but, like for me it doesn't quite line up, I'm quite happy using Pocket on Android to listen to things. It ships the text to (I think Google's) hosted text-to-speech service rather than generating it on the phone so it's higher quality. Pluses are that I already have a lot of the stuff I read in Pocket whether I'm going to listen to it or read it and I can put basically anything on the web in there.

comment by Heron (jane-mccourt) · 2021-10-19T13:51:48.153Z · LW(p) · GW(p)

This is great! To have posts so clearly read is helpful, so accessible to those of us for whom the auditory modality is stronger. Maybe specific posts, because of high interest/importance/significance/ could be read by a human, not necessarily the author, just because that adds interest.

comment by Jan Christian Refsgaard (jan-christian-refsgaard) · 2022-01-30T09:36:27.047Z · LW(p) · GW(p)

The text to speech is phenomenal!, Only math and tables suck

Suggestions for future iterations:

  1. allow filtering of the RSS by hacking the url, for example ?sources=LW,EA&quality=curated would give the curated posts from only EA and LW ignoring alignment forum
  2. somehow allow us to convert our "read later" to an RSS :)
  3. when there are figures in the post, then put them in the podcast "image" like SolenoidEntity does for the Astral Codex 10 podcast
  4. put the text in the show notes so we can pause and look at tables
Replies from: ea247
comment by KatWoods (ea247) · 2022-01-31T15:31:39.605Z · LW(p) · GW(p)

Thanks for the suggestions!

  1. We're working on setting up forum-specific channels. There's just a technical issue we're working on, then it'll happen.
  2. Check out Tayl! It does a little what you're looking for.  And you can also use @VoiceReader on Android and NaturalReader on Chrome. 
  3. Interesting idea! We want this to stay automated so we can focus on other projects and this seems likely to be hard to automate. Will look into it though!
  4. This is probably the most requested feature! We're working on it, but it's proving more difficult than we'd expected. Stay tuned. 
Replies from: jan-christian-refsgaard
comment by Jan Christian Refsgaard (jan-christian-refsgaard) · 2022-02-02T19:16:55.640Z · LW(p) · GW(p)

Thanks for the reply, 3 seams very automatable, record all text before the image, if that's 4 minuts then then put the image in after 4 min. But i totally get that stuff is more complicated than it initially seems, keep up the good work!

comment by CraigMichael · 2021-10-20T08:21:20.538Z · LW(p) · GW(p)

Would you accept volunteers to read some? Like Solenoid Entity does? 

I don't have a sexy Australian accent, I have a foxy Front Range twang. 

Replies from: parsley, PhilippaE
comment by konstell (parsley) · 2021-10-21T01:08:19.708Z · LW(p) · GW(p)

If you have the time for it, please do. I tried to listen to an episode (namely, Your Time Might Be More Valuable Than You Think) and found it more difficult to parse than a regular podcast or audiobook because of the lack of intonation. Subtle cues like tone and pauses seem to make a larger difference in my understanding than I have previously realized. That being said, I haven't experimented with the playback speed yet, maybe slowing down a bit would make the audio easier to parse.

comment by PhilippaE · 2021-10-28T05:02:18.805Z · LW(p) · GW(p)

I'd volunteer for this too! Especially others were involved in selecting texts and other organisation.

comment by CuriousApe · 2021-10-20T00:48:37.533Z · LW(p) · GW(p)

I do wonder about the positive $1000 of impact per each reader. That estimate does seem quite high to me and if it was actually the case then we should have way more people reading these articles. 

Replies from: Kenny
comment by Kenny · 2021-10-20T00:53:30.164Z · LW(p) · GW(p)

I often see people pulling random numbers out of their asses on here. No wonder there are so many college dropouts on here if that's how they go about doing research (i.e. The people who introduced me to this website are college dropouts themselves). Does the general public and established institutions take these works seriously? These articles and "research" seem to be rather self-contained in their own niche communities, much less of a public acceptance of their work at large. Makes me wonder just how much of these things are taken seriously outside of these communities themselves.

Replies from: gwern
comment by gwern · 2021-10-20T01:06:41.963Z · LW(p) · GW(p)

Here are some purely hypothetical numbers just to illustrate this way of thinking:

I often see people pulling random numbers out of their asses on here.

Replies from: Kenny
comment by Kenny · 2021-10-20T01:14:28.194Z · LW(p) · GW(p)

Not every number pulled out of the ass is annotated with that. You are arguing semantics with me while I'm just pointing out a general observation on this website. It's not that hard to verify my observations though. Seems like you are just trying to be defensive rather than providing anything of substance to this discussion. There is definitely not a lack of fictional content on here though, just saying.

comment by CuriousApe · 2021-10-20T00:25:20.095Z · LW(p) · GW(p)

You're right about all the downsides of Pocket listen feature. I use it often and it does have a lot of shortcomings. Way better on the podcast apps. 

comment by Kenny · 2021-10-20T02:32:33.334Z · LW(p) · GW(p)

How is this not a cult?

Goal: increase the number of people who read EA research

Sure you want people to read about climate change, but those things have extensive peer-reviewed research and have gone through a lot of vetting of information and their reliability. What makes the stuff on this site worthwhile? I truly want to understand why these stuff are worthwhile if people haven't put in the work to verify and vet the content. A lot of fictions and a lot of hypothetical scenarios don't usually lead to legitimacy by themselves. Does this make sense to you?