LessWrong 2.0 Reader

The examples you provided don't actually support the "one funeral at a time" narrative in your title. Take Barbara McClintock's jumping genes or Barry Marshall's H. pylori discovery -- in both cases, many scientists changed their views based on compelling evidence while very much alive. There are plenty of other examples of this. For example, the acceptance of prions as disease agents, the role of microbiomes in health, dark energy, and mitochondria's bacterial origins all show how consensus can shift rapidly once a sufficient amount of evidence has accumulated. Scientists change their minds all. the. time. 

This is not to say that there are not fads or incorrect beliefs in science -- of course there are. And sometimes it can takes years or decades for them to be overwhelmed. But the "funeral" framing in particular is not only historically inaccurate but also promotes a harmful view that death is necessary for progress. What we actually see in these examples is that scientific views change when sufficient evidence accumulates and a sufficient number of people are convinced, regardless of generational turnover. Suggesting we need scientists to die rather than be convinced by evidence is both incorrect and ethically fraught. I am saddened to see it here and therefore strong downvoted this post. 

Yeah that's fair, I'm just so used to American odds for gambling that I mentally use them all the time for these sorts of things.

Probably should have used good old fashioned odds instead.

The reason casino's show something like "Yankee's +110 Red sox -120" is so you can easily see the casino's rake or something. 

I'm pretty sure Ryan is rejecting the claim that the people hiring for the roles in question are worse-than-average at detecting illegible talent.

Depends on what you mean by "resume building", but I don't think this is true for "need to do a bunch of AI safety work for free" or similar.  i.e. for technical research, many people that have gone through MATS and then been hired at or founded their own safety orgs have no prior experience doing anything that looks like AI safety research, and some don't even have much in the way of ML backgrounds.  Many people switch directly out of industry careers into doing e.g. ops or software work that isn't technical research.  Policy might seem a bit trickier but I know several people who did not spend anything like years doing resume building before finding policy roles or starting their own policy orgs and getting funding.  (Though I think policy might actually be the most "straightforward" to break into, since all you need to do to demonstrate compentence is publish a sufficiently good written artifact; admittedly this is mostly for starting your own thing.  If you want to get hired at a "larger" policy org resume building might matter more.)

Good, I was feeling bad for Hermione. +1.

But legibility is a separate issue. If there are people who would potentially be good safety reseachers, but they get turned away by recruiters because they don't have a legibly impressive resume, then you have the companies lacking employees they would do well with if they had.

So, companies could be less constrained on people if they were more thorough in evaluating people on more than shallow easily-legible qualities.

Spending more money on this recruitment evaluation would thus help alleviate lack of good researchers. So money is tied into person-shortage in this additional way.

see also: https://www.lesswrong.com/posts/Wiz4eKi5fsomRsMbx/change-my-mind-veganism-entails-trade-offs-and-health-is-one

Re "big science": I'm not familiar with the term, so I'm not sure what the exact question being asked is. I am much more optimistic in the worlds where we have large scale coordination amongst expert communities. If the question is around what the relationship between governments, firms and academia, I'm still developing my gears around this. Jade Leung's thesis seems to have an interesting model but I have yet to dig very deep into it.

And this is how talking is anchrored in Costly Signaling.

(Note that "I dunno, probably around 9 pm." is still an assurance, though of a different kind: You're assuring that 9 pm is an honest estimate. If it turns out you make such statements up at random, it will cost you.)

And that's why talking can convey information at all.

It's neat to remember stories like this, but I want to note that this shouldn't necessarily update scientists to criticize novel work less. If an immune system doesn't sometimes overreact, it's not doing its job right, and for every story like this there are multiple other stories of genuinely false exciting-sounding ideas that got shut down by experts (for instance I learned about Schekhtman from the Constant podcast, where his story was juxtaposed with that of genuine quacks). Looking back at my experience of excited claims that were generally dismissed by more skeptical experts in fields I was following, the majority of them (for instance the superluminal neutrino, the room-temperature superconductor, various hype about potentially proving the Riemann hypothesis by well-established mathematicians) have been false.

I think there is a separate phenomenon (which was the explanation for the study about funerals), that older high-status scientists in funding-hungry fields will often continue to get funding and set priorities after they have stopped working on genuinely exciting stuff -- whether because of age, because of age-related conservatism bias, or simply because their area of expertise has become too well-developed to generate new ideas. In my experience in math and physics, from inside the field, this phenomenon generally does not look like a consensus that only the established people know what's going on (as in most of the stories here), but either conversely a quiet consensus that so-and-so famous person is starting to go crazy, or alternatively the normal disagreement between more conservative and more innovation-minded people about the value of a new idea. For example the most exciting development in my professional life as a mathematician was Jacob Lurie's development of "higher category theory", a revolution that allowed algebraists to seamlessly use tools from topology. There were many haters of this theory (many very young), but there was enough of a diffuse understanding that this is exciting and potentially revolutionary that his ideas did percolate and end up converting many of the haters (similarly with Grothendieck and schemes). Note that here I think math avoids the worst aspects of these dynamics because it doesn't require funding and is less competitive.

The upshot here is that I think it's valuable to try to resolve the issue of good ideas being shot down by traditionalists, but the solution might not be to "adopt lower standards for criticizing new / surprising ideas" but rather something more like pulling the rope sideways and looking for better standards that do better at separating promising innovation from hype.