LessWrong 2.0 Reader

http://www.overcoming-bias.com/2007/06/against_free_th.html.

This link should be: https://www.overcomingbias.com/p/against_free_thhtml (removing the hyphen will allow a successful redirect).

Most of the time, the data you gather about the world is that you have a bunch of facts about the world and probabilities about the individual data points and you would want as an outcome also probabilities over individual datapoints.

As far as my own background goes, I have not studied logic or the math behind the AI algorithm that David Chapman wrote. I did study bioinformatics in that that study we did talk about probabilities calculations that are done in bioinformatics, so I have some intuitions from that domain, so I take a bioinformatics example even if I don't know exactly how to productively apply predicate calculus to the example.

If you for example get input data from gene sequencing and billions of probabilities (a_1, a_2, ..., a_n) and want output data about whether or not individual genetic mutations exist (b_1, b_2, ..., b_m) and not just P(B) = P(b_1) * P(b_2) * ... * P(b_m).

If you have m = 100,000 in the case of possible genetic mutations, P(B) is a very small number with little robustness to error. A single bad b_x will propagate to make your total P(B) unreliable. You might have an application where getting a b_234, b_9538 and b _33889 wrong is an acceptable error because most of the values where good.

It seems like the current meta is to write a big essay outlining your opinions about AI (see, e.g., Gladstone Report, Situational Awareness, various essays recently by Sam Altman and Dario Amodei, even the A Narrow Path report I co-authored).  

Why do we think this is the case?
I can imagine at least 3 hypotheses:
1.  Just path-dependence; someone did it, it went well, others imitated

2. Essays are High Status Serious Writing, and people want to obtain that trophy for their ideas

3. This is a return to the true original meaning of an essay, under Montaigne, that it's an attempt to write thinking down when it's still inchoate, in an effort to make it more comprehensible not only to others but also to oneself.  And AGI/ASI is deeply uncertain, so the essay format is particularly suited for this.

What do you think?

An exciting recent development is community peer review, also called “open peer review.” Under this system, preprints are uploaded to a server, wherein a pool of reviewers can look them over and decide which, if any, they would like to review. Articles that have made it out of this pool are then selected for publication. This differs from the “upload PDFs to the internet” ideas because it is more structured, results in a definitive outcome, and allows gatekeeping in terms of the composition of the pool of reviewers. 

That... seems like a weird framing of what is going on? Community peer-review was the standard before anonymous and random peer review ended up being forced on the scientific institution, the way this article describes. Post-publication community peer review was the standard in most fields until the mid of the 20th century, and describing it as an exciting recent development feels like it's conceding the whole debate. 

Yes, just do post-publication peer review. Let journals and authors curate which papers they think are good at the same time as everyone else gets to read them. That's what science did before various large government funding bodies demanded more objectivity in the process (with, as this article and other articles it links to, great harm to the process of science).

Morality means sometimes not following the incentives.

I am not saying that people who always follow the incentives are immoral. Maybe they are just lucky and their incentives happen to be aligned with doing the right thing. Too much luck is suspicious though.

My main insight from all this is that we should be thinking in terms of taxonomisation of features. Some are very token-specific, others are more nuanced and context-specific (in a variety of ways). The challenge of finding maximally activating text samples might be very different from one category of features to another.

Joseph and Johnny did some interesting work on this in 'Understanding SAE Features with the Logit Lens' [LW · GW], taxonomizing features as partition features vs suppression features vs prediction features, and using summary statistics to distinguish them.

No sorry, I meant from the perspective of the person with less legible skills.

I suppose the participant is just supposed to lie about their credence here in order to "win". 

On Tuesday your credence in Heads supposed to be 0, but saying the true value would go against the experimental protocol unless you also said that your credence is 0 on Monday, which would also be a lie.

Calculations on Hydroelectric Energy Storage

For those interested in the numbers on pumped hydroelectric storage, we can get more energy by increasing 'head' or the distance that the weight falls, from 6 meters to up to 500 meters for some of the largest projects (and we could in theory go bigger).

Let's pick a more reasonable number like 60 meters:

MASS/house = 15 kWh/house / (9.8 m/s² × 60 m) = 91,836 kg/house = 91 m^3/house

Let's say we have a dam with ~20 meters of water level fluctuation (drawdown). Then that's 5 m^2 per house of surface area.

As a sanity check, Bath County Pumped Storage Station in VA stores about 24000 MWh/ 30 KWh/house = 800,000 houses worth of energy.

800,000 houses * 5 m^2 = 4 km^2

The Bath County reservoir is about 1km^2 so we're in the right range here (the reservoir has a little more drawdown and a way bigger head).

Nice, especially the second half.

To me, this post makes it seem like the natural place to push forward is to try and restrict incoherent policies over lotteries until they're just barely well-behaved - either on the behavior end (continuity etc) or the algorithmic end (compte budgets etc).