LessWrong 2.0 Reader

Does this question require that there is only one big filter per species?

on the one hand, mechanistic understanding has historically underperformed as a research strategy,

Are you talking about ML or in general? What are you deriving this from?

Well, what's the alternative? I think it might be helpful to try to make the case for doing these things via some of the alternatives:

1. a peer-reviewed Nature paper which would be published 3 years from now, maybe, behind a paywall
2. a Tiktok video
3. a 5-minute excerpted interview on CNN
4. a WSJ or NYT op-ed
5. an Arxiv paper in the standard LaTeX template
6. a Twitter thread of 500 tweets (which can only be read by logged-in users)
7. a Medium post (which can't be read because it is written in a light gray font illegible to anyone over the age of 20. Also, it's paywalled 90% of the time.)
8. interpretive dance in front of the Lincoln Memorial
9. ...

http://www.overcoming-bias.com/2007/06/against_free_th.html.

This link should be: https://www.overcomingbias.com/p/against_free_thhtml (removing the hyphen will allow a successful redirect).

Most of the time, the data you gather about the world is that you have a bunch of facts about the world and probabilities about the individual data points and you would want as an outcome also probabilities over individual datapoints.

As far as my own background goes, I have not studied logic or the math behind the AI algorithm that David Chapman wrote. I did study bioinformatics in that that study we did talk about probabilities calculations that are done in bioinformatics, so I have some intuitions from that domain, so I take a bioinformatics example even if I don't know exactly how to productively apply predicate calculus to the example.

If you for example get input data from gene sequencing and billions of probabilities (a_1, a_2, ..., a_n) and want output data about whether or not individual genetic mutations exist (b_1, b_2, ..., b_m) and not just P(B) = P(b_1) * P(b_2) * ... * P(b_m).

If you have m = 100,000 in the case of possible genetic mutations, P(B) is a very small number with little robustness to error. A single bad b_x will propagate to make your total P(B) unreliable. You might have an application where getting a b_234, b_9538 and b _33889 wrong is an acceptable error because most of the values where good.

It seems like the current meta is to write a big essay outlining your opinions about AI (see, e.g., Gladstone Report, Situational Awareness, various essays recently by Sam Altman and Dario Amodei, even the A Narrow Path report I co-authored).  

Why do we think this is the case?
I can imagine at least 3 hypotheses:
1.  Just path-dependence; someone did it, it went well, others imitated

2. Essays are High Status Serious Writing, and people want to obtain that trophy for their ideas

3. This is a return to the true original meaning of an essay, under Montaigne, that it's an attempt to write thinking down when it's still inchoate, in an effort to make it more comprehensible not only to others but also to oneself.  And AGI/ASI is deeply uncertain, so the essay format is particularly suited for this.

What do you think?

An exciting recent development is community peer review, also called “open peer review.” Under this system, preprints are uploaded to a server, wherein a pool of reviewers can look them over and decide which, if any, they would like to review. Articles that have made it out of this pool are then selected for publication. This differs from the “upload PDFs to the internet” ideas because it is more structured, results in a definitive outcome, and allows gatekeeping in terms of the composition of the pool of reviewers. 

That... seems like a weird framing of what is going on? Community peer-review was the standard before anonymous and random peer review ended up being forced on the scientific institution, the way this article describes. Post-publication community peer review was the standard in most fields until the mid of the 20th century, and describing it as an exciting recent development feels like it's conceding the whole debate. 

Yes, just do post-publication peer review. Let journals and authors curate which papers they think are good at the same time as everyone else gets to read them. That's what science did before various large government funding bodies demanded more objectivity in the process (with, as this article and other articles it links to, great harm to the process of science).

Morality means sometimes not following the incentives.

I am not saying that people who always follow the incentives are immoral. Maybe they are just lucky and their incentives happen to be aligned with doing the right thing. Too much luck is suspicious though.

My main insight from all this is that we should be thinking in terms of taxonomisation of features. Some are very token-specific, others are more nuanced and context-specific (in a variety of ways). The challenge of finding maximally activating text samples might be very different from one category of features to another.

Joseph and Johnny did some interesting work on this in 'Understanding SAE Features with the Logit Lens' [LW · GW], taxonomizing features as partition features vs suppression features vs prediction features, and using summary statistics to distinguish them.

No sorry, I meant from the perspective of the person with less legible skills.