80,000 hours should remove OpenAI from the Job Board (and similar EA orgs should do similarly)

I'm not Elizabeth or Ray, but there's a third option which I read the comment above to mean, and which I myself find plausible.

OpenAI does have roles that are obsessively aimed at reducing catastrophic risk from egregious AI misalignment. However, without more information, an outsider should not expect that those roles actually accelerate safety more than they accelerate capabilities.

Successfully increasing safety faster than capabilities requires that person to have a number of specific skills (eg political savvy, robustness to social pressure, a higher granularity strategic/technical model than most EAs have in practice, the etc.), over and above the skills that would be required to get hired for the role. 

Lacking those skills, a hire for such a role is more likely to do harm than good, not primarily because they'll be transitioned to other tasks, but because much of the work that the typical hire for such a role would end up doing either 1) doesn't help or 2) will end up boosting OpenAI's general capabilities more than it helps. 

Furthermore, by working at OpenAI at all, they provide some legitimacy to the org as a whole, and to the existentially dangerous work happening in other parts of it, even if their work, does 0 direct harm. Someone working in such a role has to do sufficiently beneficial on-net work to overcome this baseline effect.

I'm not Elizabeth and probably wouldn't have worded my thoughts quite the same, but my own position regarding your first bullet point is:

"When I see OpenAI list a 'safety' role, I'm like 55% confident that it has much to do with existential safety, and maybe 25% that it produces more existential safety than existential harm." 

Good question. My revised belief is that OpenAI will not sufficiently slow down production in order to boost safety. It may still produce theoretical safety work that is useful to others, and to itself if the changes are cheap to implement. 

I do also expect many people assigned to safety to end up doing more work on capabilities, because the distinction is not always obvious and they will have so many reasons to err in the direction of agreeing with their boss's instructions. 

 how likely do you believe it is that OAI has a position where at least 90% of people who are both, (A) qualified skill wise (eg, ML and interpretability expert), and, (B) believes that AIXR is a serious problem, would increase safety faster than capabilities in that position?

The cheap answer here is 0, because I don't think there is any position where that level of skill and belief in AIXR has a 90% chance of increasing net safety. Ability to do meaningful work in this field is rarer than that.

So the real question is how does OpenAI compare to other possibilities? To be specific, let's say being an LTFF-funded solo researcher, academia, and working at Anthropic.

Working at OpenAI seems much more likely to boost capabilities than solo research and probably academia. Some of that is because they're both less likely to do anything. But that's because they face OOM less pressure to produce anything, which is an advantage in this case. LTFF is not a pressure- or fad-free zone, but they have nothing near the leverage of paying someone millions of dollars, or providing tens of hours each week surrounded by people who are also paid millions of dollars  to believe they're doing safe work. 

I feel less certain about Anthropic. It doesn't have any of terrible signs OpenAI did (like the repeated safety exoduses, the board coup, and clawbacks on employee equity), but we didn't know about most of those a year ago.

If we're talking about a generic skilled and concerned person, probably the most valuable thing they can do is support someone with good research vision. My impression is that these people are more abundant at Anthropic than OpenAI, especially after the latest exodus, but I could be wrong. This isn't a crux for me for the 80k board^[1] but it is a crux for how much good could be done in the role.

Some additional bits of my model:

• I doubt OpenAI is going to tell a dedicated safetyist they're off the safety team and on direct capabilities. But the distinction is not always obvious, and employees will be very motivated to not fight OpenAI on marginal cases.
  • You know those people who stand too close, so you back away, and then they move closer? Your choices in that situation are to steel yourself for an intense battle, accept the distance they want, or leave. Employers can easily pull that off at scale. They make the question become "am I sure this will never be helpful to safety?" rather than "what is the expected safety value of this research?" 
  • Alternate frame: How many times will an entry level engineer get to say no before he's fired? 
• I have a friend who worked at OAI. They'd done all the right soul searching and concluded they were doing good alignment work. Then they quit, and a few months later were aghast at concerns they'd previous dismissed. Once you are in the situation is is very hard to maintain accurate perceptions. 
• Something @Buck [LW · GW] said made me realize I was conflating "produce useful theoretical safety work" with "improve the safety of OpenAI's products." I don't think OpenAI will stop production for safety reasons^[2], but they might fund theoretical work that is useful to others, or that is cheap to follow themselves (perhaps because it boosts capabilities as well...).

This is a good point and you mentioning it updates me towards believing that you are more motivated by (1) finding out what's true regarding AIXR and (2) reducing AIXR, than something like (3) shit talking OAI.

Thank you. My internal experience is that my concerns stem from around x-risk (and belatedly the wage theft). But OpenAI has enough signs of harm and enough signs of hiding harm that I'm fine shit talking as a side effect, where normally I'd try for something more cooperative and with lines of retreat. 

1. ^{^}

    I think the clawbacks are disqualifying on their own, even if they had no safety implications. They stole money from employees! That's one of the top 5 signs you're in a bad workplace. 80k doesn't even mention this.

2. ^{^}

   to ballpark quantify: I think there is <5% chance that OpenAI slows production by 20% or more, in order to reduce AIXR. And I believe frontier AI companies need to be prepared to slow by more than that. 

A statement I could truthfully say:

"As a AI safety community member, I predict I and others will be uncomfortable with 80k if this is where things end up settling, because of disagreeing. I could be convinced otherwise, but it would take extraordinary evidence at this point. If my opinions stay the same and 80k's also are unchanged, I expect this make me hesitant to link to and recommend 80k, and I would be unsurprised to find others behaving similarly."

But you did not say it (other than as a response to me). Why not? 

I'd be happy for you to take the discussion with 80k and try to change their behaviour. This is not the first time I told them that if they list a job, a lot of people will both take it as an endorsement, and trut 80k that this is a good job to apply for. 

As far as I can tell 80k is in complete denial on the large influence they have on many EAs, especially local EA community builders. They have a lot of trust, mainly for being around for so long. So when ever they screw up like this, it causes enormous harm. Also since EA have such a large growth rate (at any given time most EAs are new EAs), the community is bad at tracking when 80k does screw up, so they don't even loose that much trust.

On my side, I've pretty much given up on them caring at all about what I have to say. Which is why I'm putting so litle effort into how I word things. I agree my comment could have been worded better (with more effort), and I have tried harder in the past. But I also have to say that I find the level of extreme politeness, lot's of EA show towards high status orgs, to be very off-putting, so I never been able to imitate that style. 

Again, if you can do better, please do so. I'm serious about this.

Someone (not me) had some success at getting 80k to listen, over at the EA forum version of this post. But more work is needed.

(FWIW, I'm not the one who downvoted you)

 

You don't want to make it a new element of the menu that appears when the user clicks on the 3 vertical dots in the upper right corner of a comment?

I probably would have done something similar to a random user, though probably with a more transparent writeup, and/or trying harder to shrink the image or something.

I'll note that [censored_meme.png] is something Jacob added back in after I removed the image, not something I edited in.