Sam Altman fired from OpenAI

Update: Greg Brockman quit.

Update: Sam and Greg say:

Sam and I are shocked and saddened by what the board did today.

Let us first say thank you to all the incredible people who we have worked with at OpenAI, our customers, our investors, and all of those who have been reaching out.

We too are still trying to figure out exactly what happened. Here is what we know:

- Last night, Sam got a text from Ilya asking to talk at noon Friday. Sam joined a Google Meet and the whole board, except Greg, was there. Ilya told Sam he was being fired and that the news was going out very soon.

- At 12:19pm, Greg got a text from Ilya asking for a quick call. At 12:23pm, Ilya sent a Google Meet link. Greg was told that he was being removed from the board (but was vital to the company and would retain his role) and that Sam had been fired. Around the same time, OpenAI published a blog post.

- As far as we know, the management team was made aware of this shortly after, other than Mira who found out the night prior.

The outpouring of support has been really nice; thank you, but please don’t spend any time being concerned. We will be fine. Greater things coming soon.

Update: three more resignations including Jakub Pachocki.

Update:

Sam Altman's firing as OpenAI CEO was not the result of "malfeasance or anything related to our financial, business, safety, or security/privacy practices" but rather a "breakdown in communications between Sam Altman and the board," per an internal memo from chief operating officer Brad Lightcap seen by Axios.

Update: Sam is planning to launch something (no details yet).

Update: Sam may return as OpenAI CEO.

Update: Tigris.

Update: talks with Sam and the board.

Update: Mira wants to hire Sam and Greg in some capacity; board still looking for a permanent CEO.

Update: Emmett Shear is interim CEO; Sam won't return.

Update: lots more resignations (according to an insider).

Update: Sam and Greg leading a new lab in Microsoft.

Update: total chaos.

Also seems pretty significant:

As a part of this transition, Greg Brockman will be stepping down as chairman of the board and will remain in his role at the company, reporting to the CEO.

The remaining board members are:

OpenAI chief scientist Ilya Sutskever, independent directors Quora CEO Adam D’Angelo, technology entrepreneur Tasha McCauley, and Georgetown Center for Security and Emerging Technology’s Helen Toner.

Has anyone collected their public statements on various AI x-risk topics anywhere?

↑ comment by Zach Stein-Perlman · 2023-11-17T21:02:27.441Z · LW(p) · GW(p)

Has anyone collected their public statements on various AI x-risk topics anywhere?

A bit, not shareable.

Helen is an AI safety person. Tasha is on the Effective Ventures board. Ilya leads superalignment. Adam signed the CAIS statement. 

Replies from: shay-ben-moshe, Benito

↑ comment by ShayBenMoshe (shay-ben-moshe) · 2023-11-17T21:09:54.741Z · LW(p) · GW(p)

For completeness - in addition to Adam D’Angelo, Ilya Sutskever and Mira Murati signed the CAIS statement as well.

Replies from: daniel-kokotajlo

↑ comment by Daniel Kokotajlo (daniel-kokotajlo) · 2023-11-17T23:39:28.448Z · LW(p) · GW(p)

Didn't Sam Altman also sign it?

Replies from: habryka4

↑ comment by habryka (habryka4) · 2023-11-17T23:51:36.421Z · LW(p) · GW(p)

Yes, Sam has also signed it.

Replies from: evhub

↑ comment by evhub · 2023-11-18T09:06:13.096Z · LW(p) · GW(p)

Notably, of the people involved in this, Greg Brockman did not sign the CAIS statement, and I believe that was a purposeful choice.

↑ comment by Ben Pace (Benito) · 2023-11-17T21:51:19.674Z · LW(p) · GW(p)

Also D'Angelo is on the board of Asana, Moskovitz's company (Moskovitz who funds Open Phil).

Replies from: nikolas-kuhn

↑ comment by Amalthea (nikolas-kuhn) · 2023-11-17T22:12:13.095Z · LW(p) · GW(p)

Judging from his tweets, D'Angelo seems like significantly not concerned with AI risk, so I was quite taken aback to find out he was on the OpenAI board. This might be misinterpreting his views based on vibes.

"OpenAI’s ouster of CEO Sam Altman on Friday followed internal arguments among employees about whether the company was developing AI safely enough, according to people with knowledge of the situation.

Such disagreements were high on the minds of some employees during an impromptu all-hands meeting following the firing. Ilya Sutskever, a co-founder and board member at OpenAI who was responsible for limiting societal harms from its AI, took a spate of questions.

At least two employees asked Sutskever—who has been responsible for OpenAI’s biggest research breakthroughs—whether the firing amounted to a “coup” or “hostile takeover,” according to a transcript of the meeting. To some employees, the question implied that Sutskever may have felt Altman was moving too quickly to commercialize the software—which had become a billion-dollar business—at the expense of potential safety concerns."

Kara Swisher also tweeted:

"More scoopage: sources tell me chief scientist Ilya Sutskever was at the center of this. Increasing tensions with Sam Altman and Greg Brockman over role and influence and he got the board on his side."

"The developer day and how the store was introduced was in inflection moment of Altman pushing too far, too fast. My bet: [Sam will] have a new company up by Monday."

Apparently Microsoft was also blindsided by this and didn't find out until moments before the announcement.

"You can call it this way," Sutskever said about the coup allegation. "And I can understand why you chose this word, but I disagree with this. This was the board doing its duty to the mission of the nonprofit, which is to make sure that OpenAl builds AGI that benefits all of humanity." AGI stands for artificial general intelligence, a term that refers to software that can reason the way humans do. 
When Sutskever was asked whether "these backroom removals are a good way to govern the most important company in the world?" he answered: "I mean, fair, I agree that there is a not ideal element to it. 100%." 

https://twitter.com/AISafetyMemes/status/1725712642117898654

https://twitter.com/karaswisher/status/1725678898388553901 Kara Swisher @karaswisher

Sources tell me that the profit direction of the company under Altman and the speed of development, which could be seen as too risky, and the nonprofit side dedicated to more safety and caution were at odds. One person on the Sam side called it a “coup,” while another said it was the the right move.

Looks like Sam Altman might return as CEO. 

OpenAI board in discussions with Sam Altman to return as CEO - The Verge

I expect investors will take the non-profit status of these companies more seriously going forwards.

I hope Ilya et al. realize what they’ve done.

Edit: I think I’ve been vindicated a bit. As I expected money would just flock to for profit AGI labs, as it is poised to right now. I hope OpenAI remains a non profit but I think Ilya played with fire.

Now he’s free to run for governor of California in 2026:

I was thinking about it because I think the state is in a very bad place, particularly when it comes to the cost of living and specifically the cost of housing. And if that doesn’t get fixed, I think the state is going to devolve into a very unpleasant place. Like one thing that I have really come to believe is that you cannot have social justice without economic justice, and economic justice in California feels unattainable. And I think it would take someone with no loyalties to sort of very powerful interest groups. I would not be indebted to other groups, and so maybe I could try a couple of variable things, just on this issue.

...

I don’t think I’d have enough experience to do it, because maybe I could do like a few things that would be really good, but I wouldn’t know how to deal with the thousands of things that also just needed to happen.

And more importantly than that to me personally, I wanted to spend my time trying to make sure we get artificial intelligence built in a really good way, which I think is like, to me personally, the most important problem in the world and not something I was willing to set aside to run for office.

Prediction market: https://manifold.markets/firstuserhere/will-sam-altman-run-for-the-governo

Aside from obvious questions on how it will impact the alignment approach of OpenAI and whether or not it is a factional war of some sort, I really hope this has nothing to do with Sama's sister. Both options—"she is wrong but something convinced the OpenAI leadership that's she's right" and "she is actually right and finally gathered some proof of her claims"—are very bad. ...On the other hand, as cynical and grim as that is, sexual harassment probably won't spell a disaster down the line, unlike a power struggle among the tops of an AGI-pursuing company.

How surprising is this to the alignment community professionals (e.g. people at MIRI, Redwood Research, or similar)? From an outside view, the volatility/flexibility and movement away from pure growth and commercialization seems unexpected and could be to alignment researchers' benefit (although it's difficult to see the repercussions at this point).  While it is surprising to me because I don't know the inner workings of OpenAI, I'm surprised that it seems similarly surprising to the LW/alignment community as well.  

Perhaps the insiders are still digesting and formulating a response, or want to keep hot takes to themselves for other reasons. If not, I'm curious if there is actually so little information flowing between alignment communities and companies like OpenAI such that this would be as surprising as it is to an outsider.  For example, there seems to be many people at Anthropic that are directly in or culturally aligned with LW/rationality, and I expected the same to be true to a lesser extend for OpenAI.

I understood there was a real distance between groups, but still, I had a more connected model in my head that is challenged by this news and the response in the first day.

Someone writes anonymously, "I feel compelled as someone close to the situation to share additional context about Sam and company. . . ."

https://www.reddit.com/r/OpenAI/comments/17xoact/comment/k9p7mpv/

I wonder what changes will happen after Sam and Greg's exit.. I Hope they install a better direction towards AI safety.

↑ comment by MiguelDev (whitehatStoic) · 2023-11-18T01:22:02.841Z · LW(p) · GW(p)

I expect Sam to open up a new AI company.

Replies from: mishka

↑ comment by mishka · 2023-11-18T05:44:53.088Z · LW(p) · GW(p)

Yeah... On one hand, I am excited about Sam and Greg hopefully trying more interesting things than just scaling Transformer LLMs, especially considering Sam' answer to the last question on Nov. 1 at Cambridge Union, 1:01:45 in https://www.youtube.com/watch?v=NjpNG0CJRMM where he seems to think that more than Transformer-based LLMs are needed for AGI/ASI (in particular, he correctly says that "true AI" must be able to discover new physics, and he doubts LLMs are good enough for that).

On the other hand, I was hoping for a single clear leader in the AI race, and I thought that Ilya Sutskever was one of the best possible leaders for an AI safety project. And now Ilya vs. Sam and Greg Brockman are enemies, https://twitter.com/gdb/status/1725736242137182594, and if Sam and Greg would find a way to beat OpenAI, would they be able to be sufficiently mindful about safety?

Replies from: whitehatStoic, mishka

↑ comment by MiguelDev (whitehatStoic) · 2023-11-18T06:26:47.368Z · LW(p) · GW(p)

Hmmm. The way Sam behaves I can't see a path of him leading an AI company towards safety. The way I interpreted his world tour (22 countries?) talking about OpenAI or AI in general, is him trying to occupy the mindspaces of those countries.  A CEO I wish OpenAI has - is someone who stays at the offices, ensuring that we are on track of safely steering arguably the most revolutionary tech ever created - not trying to promote the company or the tech, I think it's unnecessary to do a world tour if one is doing AI development and deployment safely. 

(But I could be wrong too. Well, let's all see what's going to happen next.)

↑ comment by mishka · 2023-11-18T06:22:13.353Z · LW(p) · GW(p)

Interesting, how sharply people disagree...

It would be good to be able to attribute this disagreement to a particular part of the comment. Is that about me agreeing with Sam about "True AI" needing to be able to do novel physics? Or about me implicitly supporting the statement that LLMs would not be good enough (I am not really sure, I think LLMs would probably be able to create non-LLMs based AIs, so even if they are not good enough to achieve the level of "True AI" directly, they might be able to get there by creating differently-architected AIs)?

Or about having a single clear leader being good for safety? Or about Ilya being one of the best safety project leaders, based on the history of his thinking and his qualification? Or about Sam and Greg having a fighting chance against OpenAI? Or about me being unsure of them being able to do adequate safety work on the level which Ilya is likely to provide?

I am curious which of these seem to cause disagreement...

Replies from: whitehatStoic

↑ comment by MiguelDev (whitehatStoic) · 2023-11-18T06:31:15.113Z · LW(p) · GW(p)

I did not press the disagreement button but here is where I disagree:

Yeah... On one hand, I am excited about Sam and Greg hopefully trying more interesting things than just scaling Transformer LLMs,

Replies from: mishka

↑ comment by mishka · 2023-11-18T06:43:07.579Z · LW(p) · GW(p)

Do you mean this in the sense that this would be particularly bad safety-wise, or do you mean this in the sense they are likely to just build huge LLMs like everyone else is doing, including even xAI?

Replies from: whitehatStoic

↑ comment by MiguelDev (whitehatStoic) · 2023-11-18T06:50:55.277Z · LW(p) · GW(p)

I'm still figuring out Elon's xAI. 

But with regards with how Sam behaves - if he doesn't improve his framing^[1] of what AI could be for the future of humanity - I expect the same results.

 

1. ^{^}

   (I think he frames it with him as the main person that steers the tech rather than an organisation or humanity steering the tech - that's how it feels for me, the way he behaves.)

Replies from: mishka

↑ comment by mishka · 2023-11-18T07:04:03.600Z · LW(p) · GW(p)

I'm still figuring out Elon's xAI.

They released a big LLM, the "Grok". With their crew of stars I hoped for a more interesting direction, but an LLM as a start is not unreasonable (one does need a performant LLM as a component).

I think he frames it with him as the main person that steers the tech

Yeah... I thought he deferred to Ilya and to the new "superalignment team" Ilya has been co-leading safety-wise...

But perhaps he was not doing that consistently enough...

Replies from: whitehatStoic

↑ comment by MiguelDev (whitehatStoic) · 2023-11-18T07:29:37.787Z · LW(p) · GW(p)

They released a big LLM, the "Grok". With their crew of stars I hoped for a more interesting direction, but an LLM as a start is not unreasonable (one does need a performant LLM as a component).
 

I haven't played around with Grok so I'm not sure how capable or safe it is. But I hope Elon and his team of experts gets the safety problem right - as he has created companies with extraordinary achievements.  At least, Elon have demonstrated his aspirations to better humanity in other fields of sciences (Internet /Satellites, Space Exploration and EVs) and hope it translate to xAI and twitter. 

Yeah... I thought he deferred to Ilya and to the new "superalignment team" Ilya has been co-leading safety-wise...

I felt different about Ilya co-leading,  this seems to me that there's something happening inside OpenAI. When Ilya needed to co-lead the new safety direction this felt like: "something feels weird inside OpenAI and Ilya needed to co-lead the safety direction." So maybe the announcement today is related to that too.

Pretty sure there will new info from OpenAI next week or two weeks from now. Hoping it favors more safety directions - long term.

Replies from: mishka

↑ comment by mishka · 2023-11-18T07:55:02.133Z · LW(p) · GW(p)

I haven't played around with Grok so I'm not sure how capable or safe it is.

I expect safety of that to be at zero (they don't think GPT-3.5-level LLMs are a problem in this sense; besides they market it almost as an "anything goes, anti-censorship LLM").

But that's not really the issue; when a system starts being capable to write code reasonably well, then one starts getting a problem... I hope when they come to that, to approaching AIs which can create better AIs, they'll start taking safety seriously... Otherwise, we'll be in trouble...

Ilya co-leading

I thought he was the appropriately competent person (he was probably the AI scientist #1 in the world). The right person for the most important task in the world...

And the "superalignment" team at OpenAI was... not very strong. The original official "superalignment" approach was unrealistic and hence not good enough. I made a transcript of some of his thoughts, https://www.lesswrong.com/posts/TpKktHS8GszgmMw4B/ilya-sutskever-s-thoughts-on-ai-safety-july-2023-a [LW · GW], and it was obvious that his thinking was different from the previous OpenAI "superalignment" approach and much better (as in, "actually had a chance to succeed")...

Of course, now, since it looks like the "coup" has mostly been his doing, I am less sure that this is the leadership OpenAI and OpenAI safety needs. The manner of that has certainly been too erratic. Safety efforts should not evoke the feel of "last minute emergency"...

Replies from: Kaj_Sotala, whitehatStoic

↑ comment by Kaj_Sotala · 2023-11-18T20:40:54.208Z · LW(p) · GW(p)

I expect safety of that to be at zero

At least it refuses to give you instructions for making cocaine.

Replies from: Thane Ruthenis, mishka

↑ comment by Thane Ruthenis · 2023-11-18T21:33:34.253Z · LW(p) · GW(p)

Well. If nothing else, the sass is refreshing after the sycophancy of all the other LLMs.

↑ comment by mishka · 2023-11-18T22:09:43.378Z · LW(p) · GW(p)

That's good! So, at least a bit of safety fine-tuning is there...

Good to know...

↑ comment by MiguelDev (whitehatStoic) · 2023-11-18T08:09:27.399Z · LW(p) · GW(p)

But that's not really the issue; when a system starts being capable to write code reasonably well, then one starts getting a problem... I hope when they come to that, to approaching AIs which can create better AIs, they'll start taking safety seriously... Otherwise, we'll be in trouble...

Yeah, let's see where will they steer Grok.

And the "superalignment" team at OpenAI was... not very strong. The original official "superalignment" approach was unrealistic and hence not good enough. I made a transcript of some of his thoughts, https://www.lesswrong.com/posts/TpKktHS8GszgmMw4B/ilya-sutskever-s-thoughts-on-ai-safety-july-2023-a [LW · GW], and it was obvious that his thinking was different from the previous OpenAI "superalignment" approach and much better (as in, "actually had a chance to succeed")...

Yeah I agree with your analysis with the superalignment agenda, I think it's not a good use of the 20% of compute resources that they have. I even think the resource allocation of 20% on AI safety is not deep enough into the problem as I  think a 100% allocation^[1] is necessary. 

I haven't had much time studying Ilya, but I like the way he explains his arguments. I hope they (Ilya, the board and Mira or new CEO) will be better at expanding the tech than Sam is. Let's see. 

 

1. ^{^}

   I think the safest AI will be the most profitable technoloy as everyone will want to promote and build on top of it.

So I guess OpenAI will keep pushing ahead on both safety and capabilities, but not so much on commercialization? 

Typical speculations: 

• Annie Altman charges
• Undisclosed financial interests (AGI, Worldcoin, or YC)

Potentially relevant information: 

OpenAI insiders seem to also be blindsided and apparently angry at this move.

I personally think there were likely better ways to for Ilya's faction to get Sam's faction to negotiate with him, but this firing makes sense based on some reviews of this company having issues with communication as a whole and potentially having a toxic work environment. 

edit: link source now available in replies

Can someone from OpenAI anonymously spill the 🍵?

A wild (probably wrong) theory: Sam Altman announcing custom gpts was the thing that pushed the board to fire him.

customizable ai -> user can override rlhf (maybe, probably) -> we are at risk from AIs that have been finetunrd by bad actors

If he was fired for some form of sexual misconduct, we wouldn't change our views on AI risk. But the betting  seems to be that it wasn't that.

On the other hand, if the reason for his firing was something like he had access to a concerning test result, and was concealing it from the board and the government (illegal as per the executive order) then we're going to worry about what that test result was, and how bad it is for AI risk.

Worst case: this is a AI preventing itself from being shutdown, by getting the board members sympathetic to itself to fire the board members most likely to shut it down. (The "surely you could just switch it off" argument is lacking in imagination as to how an agi could prevent shutdown). Personally, low probabilty that it's this option.