"Publish or Perish" (a quick note on why you should try to make your work legible to existing academic communities)

Other communities should be moving to AF style publication, not the other way around. This is how science should be communicated; it has all the virtues of peer review without the massive downsides.

I just moved from neuroscience to publishing on LessWrong. The publishing structure here is far superior to a journal on the whole. Waiting for peer review instead of getting it in comments is an insane slowdown on the exchange of ideas.

Journal articles are discussed by experts in private. Blog posts are discussed in public in the comments. The difference in amount of analysis shared per amount of time is massive.

Issues like mathematical or other rigor are separate issues. Having tags and other sorting systems to distinguish long and rigorous work from quick writeups of simple ideas, points, and results would allow the best of both worlds.

Furthermore, we have known this for some time. In about 2003 exactly this type of publishing was suggested for neuroscience, for the above reasons - and as a way to give credit for review work. Neuroscience won't switch to it because of cultural lock-in. Don't give up your great good fortune in not being stuck in an antique system.

ah, I see!  It's an incentive problem!  So I guess your funding needs to be conditional on you producing legible outputs.

This rubs me the wrong way. Of course, you can make anyone do X, if you make their funding conditional on X. But whether you should do that, that depends on how sure you are that X is more valuable than whatever is the alternative.

There are already thousands of people out there whose funding is conditional on them producing legible outputs. Why is that not enough? What will change if we increase that number by a dozen?

Needless to say, writing papers and getting them into ML conferences is time-consuming. There's an opportunity cost. Is it worth doing despite the opportunity cost? I presume that, for the particular people you talked to, and the particular projects they were doing, your judgment was “Yes the opportunity cost was well worth paying”. And I’m in no position to disagree—I don’t know the details. But I wouldn't want to make any blanket statements. If someone says the opportunity cost is not worth it for them, I see that as a claim that a priori might be true or false. Your post seems to imply that almost everyone is making an error in the same direction, and therefore funders should put their thumb on the scale. That’s at least not obvious to me.

You seem to be suggesting that people in academia don’t read blog posts, and that blog posts are generically harder to read than papers. Both seem obviously false to me; for example, many peer-reviewed ML papers come along with blog posts, and the blog posts are intended to be the more widely accessible of the two.

Of course, blog posts can be unreadable too. Generally, I think that it's healthy for people to write BOTH (A) stuff with lots of jargon & technical details that conveys information well to people already in the know AND (B) highly-accessible stuff intended for a broader audience. (That’s what I try to do, at least.) I think it’s true and uncontroversial to say that blog posts are great for (B). I also happen to think that blog posts are great for (A).

Anyway, I think this OP isn’t particularly addressed at me (I have nothing I want to share that would fit in at an ML conference, as opposed to neuroscience), but if anyone cares I’d be happy to discuss in detail why I haven’t written any peer-reviewed papers related to my AI alignment work since I started it full-time 2 years ago, and have no immediate plans to, and what I’ve been doing instead to mitigate any downsides of that decision. It’s not even a close call; this decision seems very overdetermined from my perspective.

See also: Your posts should be on Arxiv [AF · GW]

I do agree we're leaving lots of value on the table and even causing active harm by not writing things up well, at least for Arxiv, for a bunch of reasons including some of the ones listed here. 

Cynically,[2] not publishing is a really good way to create a moat around your research... People who want to work on that area have to come talk to you, and you can be a gatekeeper. And you don't have to worry about somebody with more skills and experience coming along and trashing your work or out-competing you and rendering it obsolete...

I don't understand this part. They don't have to come talk to you, they just have to follow a link to Alignment Forum to read the research. And aren't forum posts easier to read than papers on arXiv? I feel like if the moat exists anywhere it is around academic journals which often do not make their papers freely accessible, use more cryptic writing norms and insist on using PDF which are not as user-friendly to read as webpages.

To be sure, I'm not disagreeing with your overall point. It would be great if at least the best research from Alignment Forum/LessWrong were on arXiv or in journals, and I think you're right we're leaving value on the table there. I have wondered about if someone just made it their job to do these conversions/submissions for top alignment research on the forums, because there are probably economies of scale for one person doing this vs. every researcher interrupting their work flow to learn how to jump through the hoops of paper conversion/submission.

I think your cynical take is pretty wrong, for the reasons Evan described. I'd add that because of the way academic prestige works, you are vulnerable to having your ideas stolen if you just write them up on LessWrong and don't publish them. You'll definitely get fewer citations, less recognition, etc.

I think people's stated motivations are the real motivations: Jumping through hoops to format your work for academia has opportunity costs and they don't judge those costs to be worth it.

I like this because it makes it clear that legibility of results is the main concern. There are certain ways of writing and publishing information that communities 1) and 2) are accustomed to. Writing that way both makes your work more likely to be read, and also incentivizes you to state the key claims clearly (and, when possible, formally), which is generally good for making collaborative progress.

In addition, one good thing to adopt is comparing to prior and related work; the ML community is bad on this front, but some people genuinely do care. It also helps AI safety research to stack.

To avoid this comment section being an echo chamber: you do not have to follow all academic customs. Here is how to avoid some of the harmful ones that are unfortunately present:

• Do not compromise on the motivation or related work to make it seem less weird for academics. If your work relies on some LW/AF posts, do cite them. If your work is intended to be relevant for x-risk, say it.
• Avoid doing anything if the only person you want to appease with it is an anonymous reviewer. 
• Never compromise on the facts. If you have results that say some famous prior paper is wrong or bad, say it loud and clear, in papers and elsewhere. It doesn't matter who you might offend.
• AI x-risk research has its own perfectly usable risk sheet you can include in your papers.
• And finally: do not publish potentially harmful things just because it benefits science. Science has no moral value. Society gives too much moral credit to scientists in comparison to other groups of people.

I think this describes how Eliezer's grudge against academia has set back AI alignment (even the parts that aren't related to his organization, since his cultural influence has made this a wider norm).

It's important to internalize that the intellectual world lives in the attention economy, like eveything else.

Just like "content creators" on social platforms think hard about capturing and keeping attention, so do intellectuals and academics.  Clarity and rigor is a part of that.

No one has time, energy, (or crayons, as the saying goes) for half-baked ramblings on a blog or forum somewhere.

Strongly agree. I went in a similar direction here: https://www.lesswrong.com/posts/Tw944k5t6tq82CFNm/portia-s-shortform?commentId=H9ffSnAasjwz8CRRc#H9ffSnAasjwz8CRRc

I hate the academic publishing industry. There is so much about it that is utterly broken. The publishing fees, the access fees, the fucking closed access, the massive delay to publication, the pressure to follow hypes and push out quantity, the pressure to stick to tiny clearly solvable problems rather than bigger and more important underlying issues, to knock out tons of tiny irrelevant papers rather than one proper book, the targeting of subfields rather than an interdisciplinary scope, the fact that peer review is often really not the objective measure of improvement that it should be, but just a way for the reviewer to force your to cite their unrelated stuff and avoid statements they dislike... and yet, the basic idea of peer review and minimum standards and easily citable work and pressure to make things mathematically precise has a hell of a lot going for it. I've shown articles from here to people working in machine learning, and they basically went "never heard of the author, I assume there is a reason for that, and that my journals have reasons for rejecting them" or "this is not tied back to code and math sufficiently for me to judge it at all; I can't even say it is wrong, I am not even sure what it is supposed to mean". They often get the impression that they need to do a lot of work to turn your paper into something meaningful and useful, and they see no reason to do so for you.

Pragmatically, it is impossible to be heard in scientific circles, where you want and need to be heard, if you do not publish. It disqualifies you from jobs, from funding, from conferences, from simply being read. They won't be aware you exist, because you do not turn up in their journals, and if they learn that you exist, they likely still won't engage, because they think that if you had a concrete point and were serious about it, you would have written it in a paper by a journal they trust and that has vetted it, and you haven't, so you either can't, lacking skills they consider basic, or you don't respect them and the topics and approaches they work on, so they do not feel like respecting yours.

I am not saying only consider academic metrics in your actions; after all, academia is not doing well on the questions we care about here. I think pushing out text outside of academic journals, to fellow interested parties, but also to the general public, is fucking important. I think engaging with big questions is important. 

But you need the grounding from getting feedback from scientists. They will often be able to tell you very quickly that a cool idea you had simply is not practical on a technical level. Or may point out that they are aware of this concern of yours, though their terminology is different and they are not debating it publicly. Also, if you want to make a difference, you need them to change what they are doing.

I know it is frustrating, and not all of these standards are justified and fair. But they are not arbitrary, they do reflect underlying important standards, and they simply have significant real world importance.

I think this post makes an important and still neglected claim that people should write their work more clearly and get it published in academia, instead of embracing the norms of the narrower community they interact with. There has been significant movement in this direction in the past 2 years, and I think this posts marks a critical change in what the community suggests and values in terms of output.

As someone with an academic background now working in mainstream ML research, I strongly endorse the message of this post.

I agree with many on this forum that (a) there is some extra work in writing an article in an "academic" style, and (b) academic articles are often written with the objective of impressing a reviewer rather than being completely transparent about what was achieved in the work and how it advances the state of knowledge.  I was incredibly frustrated by both of these issues when I was a grad student.  However, when I try to read research written on LessWrong, it often seems even more difficult to understand.  I've heard from others working in ML but not well-versed in safety that they have had similar experiences.  

Another commenter complained about papers that "getting them into ML conferences is time-consuming."  Definitely agree, and conference reviewing in ML is atrocious.  But you can avoid all of that work just by putting your work on arXiv instead of aiming for an ML conference.

This sounds like there's a niche for technical writers who format/edit AF posts as proper ML articles for credits, points and glory

Folks commenting here might want to look at this post from April 2020, Blogging is dead, long live sites, which I found in one of Tyler Cowen's link posts. The author starts of discussing the blogosphere, which he claims peaked in 2009, then moves on to essays vs. blogposts. and ends with independent vs. academic scholarship:

A lot of the sites I list in the accordion above are by part-time autodidacts, or retired scholars. I suppose this is because the incentives in academia are so often towards either small publons or giant monographs, each of which are set in stone once done. (Unless they are grossly flawed enough to trigger academia’s slow, dumb immune system.) (Or my search process is biased towards lone wolves.)

But the average academic work is more lasting than the average internet work. But it isn’t only durability we’re after. But it’s also more rigorous than the average internet work.

Robin Hanson has spotted a trend among independent scholars, a systematic bias against rigour, and so against durability.

"over time amateurs blow their lead by focusing less and relying on easier, more direct methods. They rely more on informal conversation as analysis method, they prefer personal connections over open competitions in choosing people, and they rely more on a perceived consensus among a smaller group of fellow enthusiasts. As a result, their contributions just don’t appeal as widely or as long."

Take Hanson himself: he has about 100 academic publications, two big books, and something like 3000 blog posts. Which will be his biggest contribution in the end?

Maybe tenured academics are the people best placed to do long content: lots of time, lots of connections, some pressure towards rigour and communicability. But you should be able to do it outside uni, if you’re wary.