LessWrong 2.0 Reader

As a non-profit it is obligated to not take opportunities to profit, unless those opportunities are part of it satisfying its altruistic mission.

In your initial post, it sounded like you were trying to say:

This grant was obviously ex ante bad. In fact, it's so obvious that it was ex ante bad that we should strongly update against everyone involved in making it.

I think that this argument is in principle reasonable. But to establish it, you have to demonstrate that the grant was extremely obviously ex ante bad. I don't think your arguments here come close to persuading me of this. 

For example, re governance impact, when the board fired sama, markets thought it was plausible he would stay gone. If that had happened, I don't think you'd assess the governance impact as "underwhelming". So I think that (if you're in favor of sama being fired in that situation, which you probably are) you shouldn't consider the governance impact of this grant to be obviously ex ante ineffective.

I think that arguing about the impact of grants requires much more thoroughness than you're using here. I think your post has a bad "ratio of heat to light": you're making a provocative claim but not really spelling out why you believe the premises. 

In case people missed this, another safety researcher recently left OpenAI: Ryan Lowe.

I don't know Ryan's situation, but he was a "research manager working on AI alignment."

I think your model is a bit simplistic. METR has absolutely influenced the behavior of the big labs, including DeepMind. Even if all impact goes through the big labs, you could have more influence outside of the lab than as one of many employees within. Being the head of a regulatory agency that oversees the labs sets policy in a much more direct way than a mid level exec within the company can.

I'm confused by what you mean that GPT-4o is bad? In my experience it has been stronger than plain GPT-4, especially at more complex stuff. I do physics research and it's the first model that can actually improve the computational efficiency of parts of my code that implement physical models. It has also become more useful for discussing my research, in the sense that it dives deeper into specialized topics, while the previous GPT-4 would just respond in a very handwavy way. 

It's not an entirely unfair characterization.

(Re: Difficult to Parse react on the other comment [LW(p) · GW(p)]
I was confused about relevance of your comment above [LW(p) · GW(p)] on chunky innovations, and it seems to be making some point (for which what it actually says is an argument), but I can't figure out what it is. One clue was that it seems like you might be talking about innovations needed for superintelligence, while I was previously talking about possible absence of need for further innovations to reach autonomous researcher chatbots, an easier target. So I replied with formulating this distinction and some thoughts on the impact and conditions for reaching innovations of both kinds. Possibly the relevance of this was confusing in turn.)

Aren't these different things? Private yes, for profit no. It was private because it's not like it was run by the US government.

The links/graphics are broken btw. Would probably be nice to fix if it's quick.

I think there's a solid case for anyone who supported funding OpenAI being considered at best well intentioned but very naive. I think the idea that we should align and develop superintelligence but, like, good, has always been a blind spot in this community - an obviously flawed but attractive goal, because it dodged the painful choice between extinction risk and abandoning hopes of personally witnessing the singularity or at least a post scarcity world. This is also a case where people's politics probably affected them, because plenty of others would be instinctively distrustful of corporation driven solutions to anything - it's something of a Godzilla Strategy after all, aligning corporations is also an unsolved problem - but those with an above average level of trust in free markets weren't so averse.

Such people don't necessarily have conflicts of interest (though some may, and that's another story) but they at least need to drop the fantasy land stuff and accept harsh reality on this before being of any use.