Remote AI Alignment Overhang?

post by tryactions · 2023-02-19T22:30:47.530Z · LW · GW · 5 comments

Contents

  The Problem (if there is one)
  Brainstorming Solutions
  Conclusion
None
5 comments

Background link: Do any AI alignment orgs hire remotely? [LW · GW]

Disclaimer:  The generator [LW · GW] of me writing this post is my own desire for remote AI Alignment opportunities to exist.  I have Alignment research in mind while writing this, but similar arguments might apply to other Alignment-related jobs?

This is a post about a capabilities overhang [LW · GW], but not in the usual sense.

The Problem (if there is one)

It seems to me that there's been substantial growth in ways to get involved with Alignment.  However, there are very few advertised jobs in Alignment that do not involve moving to specific locations, mostly the Bay Area. I hypothesize that people who might otherwise be productive are dropping out of Alignment, or never engaging at all, because they're not willing to move.

Who are these talented people I'm hypothesizing, and why aren't they willing to move? They might:

1. Have weak or non-existent social support networks that make risk-taking riskier.
2. Have responsibilities they've taken on (spouse/kids/mortgage/elderly parents) that give them less freedom to move.
3. Have non-prestigious backgrounds, such that they might have trouble finding work if they find themselves out of a job.
4. Have little money, such that they have less runway to reverse course if a move doesn't pan out.
5. Have an established lifestyle that they don't want to change.

Overall, I could summarize this as having less spare resources or being more firmly rooted somewhere that's not the Bay Area.  I'm pretty sure this is going to be strongly correlated with being born in a lower economic class -- you'll have less resources to get started in life, and by the time you do have resources (if ever), you're more likely to have rooted in a less expensive location outside of a major city.  Being rooted outside a major city also makes it less likely you'll develop prestigious connections.

Although it might suck for those people, this lack of opportunity isn't necessarily an important problem to solve if your goal is improving Alignment.  The rest of this article is me unpacking my thoughts about why I think it might be important.

Brainstorming Solutions

Two ideas for improving the situation would be:
1. Create alignment organizations in more cities, or
2. Create/convert alignment organizations to allow remote work.

Creating organizations in new cities has natural limits.  In a world with 300 total AI Alignment researchers [LW · GW], it's unrealistic to expect organizations in more than few major cities.  And even if a candidate is lucky enough to have an organization in their city, they may not find that particular organization to be a good fit.  A career path the depends on being hired by a single organization is also extremely risky.

As for remote work, the clearest reason for an organization not to adopt it is that productivity increases coming from a wider talent pool might be offset by remote work being less efficient.  This could be individuals being less efficient, e.g. I write less code when not on-site, or organizations being less efficient, e.g. our non-profit does the wrong research if we're not all on-site to strategize.

As a perhaps misinformed outsider to Alignment and EA professional circles, I get the impression that Alignment is considered to be mostly talent-constrained and not funding-constrained -- that is, if funders could confidently identify promising people, then they'd be willing to spend more on funding those people.  They might do this by directly funding a promising founder, or by having a founder/organization identify promising people and then passing that request on to funders via organizational fundraising.

How does this identification currently take place?  I'd appreciate correction from people who actually make such decisions, but my current understanding is that candidates are primarily assessed on:

1. Prior research success [LW · GW].
2. General markers of prestige and success outside of Alignment that might transfer -- e.g. success in research in another field, a high-level role in a successful company, or credentials from a top school.

Both of these criteria will select against the low-resource remote-only persona I presented above:

1. They don't have the resources to spare doing research that's unlikely to pay off for their career, and that career can't be in Alignment since there's no remote Alignment jobs.
2. By hypothesis, they don't have as many traditional markers of success and prestige.

Thus, even if a talented remote-only candidate does somehow come to attention, funders and founders might not be correctly identifying them as high potential.  This might lead to them underestimating the untapped remote-only talent pool.

If that talent pool contains enough gems, then it may be worth tapping into even if remote work is somewhat inefficient.  The most talented members of that pool may still make a major impact, given the chance.  They might also be willing to move for senior roles with higher career security, but not for the junior roles they would need to enter the field.

If full-time remote Alignment jobs were widely available, then there would be greater incentive for these people to spend time trying their hand at Alignment research (as opposed to e.g. grinding LeetCode).  Thus, creating/converting-to a remote-compatible Alignment organization has a secondary effect of encouraging more of this talent pool to signal its talent by by engaging with research.

Some other avenues to identify and draw out more of these people might be:

• Creating early engagement opportunities / onboarding programs that are compatible with simultaneous full-time employment.
• Relying on work samples for evaluation, as opposed to prestige and prior-success indicators.

Conclusion

I don't really know how big and talented this pool of won't-move-but-want-to-contribute people is.  But given that a large number of people never move, my prior is that it's reasonably large.  

I have the perception that funding and hiring decisions are dominated by "elites": people from prestigious schools, with supportive/wealthy/connected/knowledgeable families, and impressive achievements on their resume.   My perception is also that, on average, elites tend to underestimate how much implicit safety they have to take risks relative to non-elites.  Thus I expect there to be a natural bias in such decisions against those who are unwilling to take a risk like moving to the Bay.  

But this really is a huge risk for some people!  The Bay is literally one of the most expensive places in the world to live!  Moving can destroy ~all of someone's social capital!  It seems pretty likely to me there's some untapped talent that's not willing to take that level risk, but would happily pursue Alignment as a career all else equal.

I'd like remote Alignment research jobs to exist, and I think there's a good chance that there's talent available to fill them up that might not yet have been identified.  If you've got funding to spare and are sufficiently tolerant of risk yourself, consider creating them?

5 comments

Comments sorted by top scores.