LessWrong 2.0 Reader

Did they have any points that you found especially helpful, surprising, or interesting? Anything you think folks in AI policy might not be thinking enough about?

(Separately, I hope to listen to these at some point & send reactions if I have any.)

I'm here to say, this is not some property specific to p-values, just about the credibility of the communicator.

If $scientists$ make a bunch of errors all the time, especially those that change their conclusions, indeed you can't trust them. Turns out (BW11) that $scientist{s}_{publishedinbetterjournals}$ are more credible than $scientist{s}_{publishedinworsejournals}$, the errors they make tend not to change the conclusions of the test (i.e., the chance of drawing a wrong conclusion from their data ("gross error" in BW11) was much lower than the headline rate), and (admittedly I'm going out on a limb here) it is very possible the errors that change the conclusion of a particular test do not change the overall conclusion about the general theory (e.g., if theory says X, Y, and Z should happen, and you find support for X and Y and marginal-support-now-not-significant-support-anymore for Z, the theory is still pretty intact unless you really care about using p-values in a binary fashion. If theory says X, Y, and Z should happen, and you find support for X and Y and now-not-significant-support-anymore for Z, that's more of an issue. But given how many tests are in a paper, it's also possible theory says X, Y, and Z should happen, and you find support for X and Y and Z, but turns out your conclusion about W reverses, which may or may not really have something to say about your theory).

I don't think it is wise to throw the baby out with the bathwater.

But I also find my own understanding to be a bit confused and in need of better sources.

Mine too, for sure.

And agreed, Chollet's points are really interesting. As much as I'm sometimes frustrated with him, I think that ARC-AGI and his willingness to (get someone to) stake substantial money on it has done a lot to clarify the discourse around LLM generality, and also makes it harder for people to move the goalposts and then claim they were never moved).

Thinking about responsible gambling, something like up-front long-term commitment should solve a lot of problems? You have to decide right away and lock up money you going to spend this month and that will separate decision from impulse to spend.

Those outcomes sound quite plausible.

I'm particularly concerned with polarization. Becoming a political football was the death knell for sensible discussion on climate change, and it could be the same for AGI x-risk. Public belief in climate change actually fell while the evidence mounted. My older post AI scares and changing public beliefs [LW · GW] is actually mostly about polarization.

Having the debate become ideologically/politically motivated seems like it wouldn't be good. I'm still really hoping to avoid polarization on AGI x-risk. It does seem like "AI safety", concerns about bias, deepfakes, and harms from interacting with LLMs are already primarily discussed among liberals in the US.

Neither side has started really worrying about job loss, but that would tend to be the liberal side, too, since conservatives are still somewhat more free-market oriented.

While tying concerns about x-risk with calls to slow AI based on mundane harms might seem expedient, I wouldn't take that bargain if it created worse polarization.

I think this is a common attitude among the x-risk worried, especially since it's hard to predict whether a slowdown in the US AGI push would be a net good or bad thing for x-risk.

For what it's worth, the human brain (including the cortex) has a fixed modularity. Long range connections are created during fetal development according to genetic rules, and can only be removed, not rerouted or added to.

I believe this is what causes the high degree of functional localization in the cortex.

Is this a Lisp-to-Python transpiler? 

You might want to know that I took a look through the site, and was curious, but I just closed the page the moment the “Calculate your contribution” form refused to show me the pricing options unless I gave it an email address.

I agree with your frustrations, I think his views are somewhat inconsistent and confusing. But I also find my own understanding to be a bit confused and in need of better sources.

I do think the discussion François has in this interview is interesting. He talks about the ways people have tried to apply LLMs to ARC, and I think he makes some good points about the strengths and shortcomings of LLMs on tasks like this.

Another working job market economics paper out of Stanford attempts to measure the degree to which sports bettors are overly optimistic. Results largely what you'd expect: people think they're break even when they're actually losing by ~7% and a subset of those people have self control problems.

Funnily enough the way I found out about this paper is from being recruited to participate in it through a targeted ad on social media when I took a trip out to Colorado to farm sports book new account sign up bonuses.