Hiring engineers and researchers to help align GPT-3

post by paulfchristiano · 2020-10-01T18:54:23.551Z · LW · GW · 13 comments

Contents

  What the team does
  What you’d do
None
13 comments

My team at OpenAI, which works on aligning GPT-3, is hiring ML engineers and researchers. Apply here for the ML engineer role and here for the ML researcher role.

GPT-3 is similar enough to "prosaic" AGI that we can work on key alignment problems without relying on conjecture or speculative analogies. And because GPT-3 is already being deployed in the OpenAI API, its misalignment matters to OpenAI’s bottom line — it would be much better if we had an API that was trying to help the user instead of trying to predict the next word of text from the internet.

I think this puts our team in a great place to have an impact:

We have the real-world problems to motivate alignment research, the financial support to hire more people, and a research vision to execute on. We are bottlenecked by excellent researchers and engineers who are excited to work on alignment.

What the team does

In the past Reflection focused on fine-tuning GPT-3 using a reward function learned from human feedback. Our most recent results are here, and had the unusual virtue of simultaneously being exciting enough to ML researchers to be accepted at NeurIPS while being described by Eliezer [LW(p) · GW(p)] as “directly, straight-up relevant to real alignment problems.”

We’re currently working on three things:

If successful, ideas will eventually move up this list, from the conceptual stage to ML prototypes to real deployments. We’re viewing this as practice for integrating alignment into transformative AI deployed by OpenAI or another organization.

What you’d do

Most people on the team do a subset of these core tasks:

If you are excited about this work, apply here for the ML engineer role and here for the ML researcher role.

13 comments

Comments sorted by top scores.

comment by seed · 2020-10-01T21:03:43.437Z · LW(p) · GW(p)

Does it make sense to apply if I'm Russian? What do you think is the chance of Trump allowing H1B visas next year? Will you even consider foreign applicants? Do you provide green cards?

Replies from: Lambda, Pongo
comment by Lambda · 2020-10-02T19:43:02.139Z · LW(p) · GW(p)

Trump

Let's not get ahead of ourselves, friend.

comment by Pongo · 2020-10-05T23:35:16.817Z · LW(p) · GW(p)

It is at least the case that OpenAI has sponsored H1Bs before: https://www.myvisajobs.com/Visa-Sponsor/Openai/1304955.htm

comment by paulfchristiano · 2020-10-05T19:28:18.441Z · LW(p) · GW(p)

described by Eliezer as “directly, straight-up relevant to real alignment problems.”

Link to thread.

Worth saying that Eliezer still thinks our team is pretty doomed and this is definitely not a general endorsement of our agenda. I feel excited about our approach and think it may yet work, but I believe Eliezer's position is that we're just shuffling around the most important difficulties into the part of the plan that's vague and speculative.

I think it's fair to say that Reflection is on the Pareto frontier of {plays ball with MIRI-style concerns, does mainstream ML research}. I'm excited for a future where either we convince MIRI that aligning prosaic AI is plausible, or MIRI convinces us that it isn't.

comment by Lambda · 2020-10-02T07:17:41.799Z · LW(p) · GW(p)
  • how suitable is the research engineering job for people with no background in ml, but who are otherwise strong engineers and mathematicians?
  • will these jobs be long-term remote? if not, on what timeframe will they be remote?
Replies from: paulfchristiano
comment by paulfchristiano · 2020-10-05T19:07:20.831Z · LW(p) · GW(p)

will these jobs be long-term remote? if not, on what timeframe will they be remote?

We expect to be requiring people to work from the office again sometime next year.

how suitable is the research engineering job for people with no background in ml, but who are otherwise strong engineers and mathematicians?

ML background is very helpful. Strong engineers who are interested in learning about ML are also welcome to apply though no promises about how well we'll handle those applications in the current round.

comment by magfrump · 2020-10-03T16:19:23.002Z · LW(p) · GW(p)

What is the expected time frame of the openings?

I am personally indisposed until ~end of October and may not be ready to start a new job for a little while after that, but would otherwise be very excited for such a role.

Somewhat related, do you have an idea of how many openings there will be? Like, fewer than 3 or more than 20, for example?

Replies from: paulfchristiano
comment by paulfchristiano · 2020-10-05T19:05:19.063Z · LW(p) · GW(p)

The team is currently 7 people and we are hiring 1-2 additional people over the coming months.

I am optimistic that our team and other similar efforts will be hiring more people in the future and continuously scaling up, and that over the long term there could be a lot of people working on these issues.

(The post is definitely written with that in mind and the hope that enthusiasm will translate into more than just hires in the current round. Growth will also depend on how strong the pool of candidates is.)

comment by NicholasKross · 2020-10-04T02:05:37.211Z · LW(p) · GW(p)

What's the quickest way to get up to speed and learn the relevant skills, to the level expected of someone working at OpenAI?

Replies from: rohinmshah
comment by Rohin Shah (rohinmshah) · 2020-10-05T04:33:28.976Z · LW(p) · GW(p)

Relevant 80K podcast: https://80000hours.org/podcast/episodes/olsson-and-ziegler-ml-engineering-and-safety/

comment by jbash · 2020-10-02T13:40:14.500Z · LW(p) · GW(p)

it would be much better if we had an API that was trying to help the user instead of trying to predict the next word of text from the internet.

"I'm from OpenAI, and I'm here to help you".

Seriously, it's not obvious that you're going to do anything but make things worse by trying to make the thing "try to help". I don't even see how you could define or encode anything meaningfully related to "helping" at this stage anyway.

As for the bottom line, I can imagine myself buying access to the best possible text predictor, but I can't imagine myself buying access to something that had been muddied with whatever idea of "helpfulness" you might have. I just don't want you or your code making that sort of decision for me, thanks.

Replies from: Benito, paulfchristiano
comment by Ben Pace (Benito) · 2020-10-02T18:54:42.051Z · LW(p) · GW(p)

(Upvoted, because jbash is a good commenter and it's a pretty reasonable question for someone unacquainted with Paul's work.)

Hey jbash. So, while you're quite right in the short term that in general the 'helpful' bots we build are irritating and inflexible (e.g. Microsoft's Clippy), the main point of a lot of Paul's AI research is to figure out how to define helpfulness in such a way that an ML system can successfully be trained to do it – the hard problem of defining 'helpfulness', not the short term version of "did a couple of users say it was helpful and did the boss say ship it". He's written about it in this [? · GW] post, and given a big-picture motivation for it here [LW · GW].

It's abstract and philosophically hard and it's quite plausibly will just not work out, but I do think Paul is explicitly attempting to solve the hard version of the problem with the full knowledge of what you said.

comment by paulfchristiano · 2020-10-05T19:19:02.180Z · LW(p) · GW(p)

I think that "imitate a human who is trying to be helpful" is better than "imitate a human who is writing an article on the internet," even though it's hard to define "helpful." I agree that's not completely obvious for a bunch of reasons.

(GPT-3 is better if your goal is in fact to predict text that people write on the internet, but that's a minority of API applications.)