MATS Alumni Impact Analysis

post by utilistrutil, Juan Gil (Zwischnu), yams (william-brewer), LauraVaughan (laura-vaughan), K Richards, Ryan Kidd (ryankidd44) · 2024-09-30T02:35:57.273Z · LW · GW · 6 comments

Contents

  Summary
  Background on Cohort
  Employment Outcomes
  Publication Outcomes
  Other Outcomes
  Evaluating Program Elements
  Career Plans
  Acknowledgements
None
6 comments

Summary

This winter, MATS will be running our seventh program. In early-mid 2024, 46% of alumni from our first four programs (Winter 2021-22 to Summer 2023) completed a survey about their career progress since participating in MATS. This report presents key findings from the responses of these 72 alumni.

Background on Cohort

For 40% of respondents, their highest academic degree was a Bachelor’s; 40% had earned at most a Master’s, and 20%, a PhD.

Their most common categories of current work were “Working/interning on AI alignment/control” (49%) and “Conducting alignment research independently” (29%).

Here are some representative descriptions of the work alumni were doing:

Three alumni who completed the full MATS program before Winter 2023-24 selected “working/interning on AI capabilities.” They described their current work as:

Erratum: previously, this section listed five alumni currently “working/interning on AI capabilities." This included two alumni who did not complete the program before Winter 2023-24, which is outside the scope of this report. Additionally, two of the three alumni listed above first completed our survey in Sep 2024 and were therefore not included in the data used for plots and statistics. We include them here for full transparency.

Employment Outcomes

We asked alumni about their career experiences since graduating from MATS. 54% of respondents applied to a job and advanced past the first round of interviews.

We asked about a range of possible outcomes from these job application processes. Among those respondents who shared details, the most common outcome for alumni who made it past the first round of interviews was accepting a job offer (64%). Accepted jobs included:

We inquired about whether MATS contributed to these alumni’s progress through the job rounds.

For 8% of alumni, MATS did not contribute to their job progression. For the others, MATS helped them build career capital and made them more likely to apply. We asked more specifically how MATS benefited alumni in these hiring rounds:

For 49% of alumni, MATS increased their research or technical skills, and for 38% of alumni, MATS provided legible career capital.

Publication Outcomes

We asked alumni about their publication records. 68% had published alignment since or during MATS.

 

The most common category of publication was a LessWrong post (45%). Conference and journal papers included:

Other publications included:

We asked about how MATS contributed to these outcomes.

For 78% of respondents, their publication “possibly” or “probably” would not have happened without MATS. 10% of alumni reported that MATS accelerated publication by more than 6 months; 14% said 1-6 months. 8% of alumni responded that MATS resulted in a “much higher” quality of their publication.

Other Outcomes

We also asked alumni:

Are there other impactful outcomes of MATS that you want to tell us about? This category is intentionally flexible. I expect that most respondents won't submit anything in this category.

Examples of outcomes that might be logged here:

  • Founding a new AI alignment organization;
  • Volunteering for an AI alignment field-building initiative (resulting in significant counterfactual impact).

Of our respondents, 20% submitted an additional impactful outcome. Three of these 16 respondents include two impactful outcomes. Multiple alumni mentioned starting new research organizations to tackle a specific AI safety research agenda. Here is a selection of responses, and how MATS influenced them:

Evaluating Program Elements

As noted in our Winter 2023-24 Retrospective [LW(p) · GW(p)], our alumni have often reported that they were willing to participate in MATS for a lower stipend than they received, which in Summer 2023 was $4800/month. 30% of alumni would have been willing to participate in MATS for no stipend at all.

We asked alumni to rate the value they got out of various program elements. 

As we observed in the Winter 2023-24 cohort [LW · GW], “mentorship” and “peers” were considered the most valuable program elements. 72% of alumni called mentorship the “most valuable part of MATS” or a “significant portion of value added” and 82% said the same of their MATS peers.

The mentors who provided less value were generally those who had very limited time to spend on their scholars or had less experience mentoring. The mentors who provided the most value had more experience and time to dedicate to their scholars, including multiple safety researchers at scaling labs.

Alumni elaborated on the value MATS provided them:

The program element “Other connections” could include collaborators, advisors, and other researchers in the Bay Area AI safety community that scholars met at MATS networking events. We asked alumni what kind of connections they made at MATS.

The most common type of connection was a research collaborator (63%), followed by a mentor/adviser (54%). 15% of alumni reported making no connections in the categories we asked about. Alumni elaborated on the value of these connections:

A few alumni offered testimonials about their experiences at MATS:

We also asked alumni whether MATS had played a negative role in any of the career outcomes they reported. Specifically, we asked “Did MATS have a negative effect on any of the outcomes (job offers, publications, etc.) you've indicated? e.g. career capital worse than counterfactual, publications worse than counterfactual, etc.” Many alumni simply responded “no” or affirmed the value MATS provided them. Others offered useful criticism:

Career Plans

We asked alumni about their future career plans:

At a high level, what's your career plan for the foreseeable future? You likely have uncertainty, so feel free to indicate your options and considerations. Generally we want to know:

  • What kind of work broadly?
  • What's your theory of change?
  • How are you trading off between immediate impact and building experience?

Don't spend more than a few minutes on this (<5 min).

Many respondents are pursuing academia or considering government and policy work. An interest in mechanistic interpretability was common among those continuing with research. Here is a selection of responses:

Alumni informed us about the types of support that might be valuable to them at their current career stages.

For 46% of alumni, connections to research collaborators would be valuable, and 39% would benefit from job recommendations. The median alum from Summer 2023 met 6 potential collaborators [LW(p) · GW(p)] during the program, but these alumni results indicate that MATS can go further in supporting our scholars with networking opportunities. Likewise, we hosted a career fair [LW · GW] during the past four programs, but these results show that MATS can provide further job opportunities to our alumni.

Acknowledgements

This report was produced by the ML Alignment & Theory Scholars Program. @utilistrutil [LW · GW] was the primary author of this report, Juan Gil and @yams [LW · GW] contributed to editing, Laura Vaughan and Kali Richards contributed to data analysis, and Ryan Kidd scoped, managed, and edited the project. Thanks to our alumni for their time and feedback! We also thank Open Philanthropy, DALHAP Investments, the Survival and Flourishing Fund Speculation Grantors, Craig Falls, Foresight Institute, and several generous donors on Manifund, without whose donations we would be unable to run upcoming programs or retain team members essential to this report.

To learn more about MATS, please visit our website. We are currently accepting donations for our Summer 2025 Program and beyond!

6 comments

Comments sorted by top scores.

comment by gw · 2024-09-30T08:14:19.024Z · LW(p) · GW(p)

Do you have any data on whether outcomes are improving over time? For example, % published / employed / etc 12 months after a given batch

Replies from: ryankidd44
comment by Ryan Kidd (ryankidd44) · 2024-09-30T21:37:58.039Z · LW(p) · GW(p)

Great suggestion! We'll publish this in our next alumni impact evaluation, given that we will have longer-term data (with more scholars) soon.

comment by aysja · 2024-09-30T23:01:56.132Z · LW(p) · GW(p)

Does the category “working/interning on AI alignment/control” include safety roles at labs? I’d be curious to see that statistic separately, i.e., the percentage of MATS scholars who went on to work in any role at labs.

Replies from: ryankidd44
comment by Ryan Kidd (ryankidd44) · 2024-09-30T23:33:17.311Z · LW(p) · GW(p)

Scholars working on safety teams at scaling labs generally selected "working/interning on AI alignment/control"; some of these also selected "working/interning on AI capabilities", as noted. We are independently researching where each alumnus ended up working, as the data is incomplete from this survey (but usually publicly available), and will share separately.

comment by DusanDNesic · 2024-10-02T05:59:53.856Z · LW(p) · GW(p)

Amazing write-up, thank you for the transparency and thorough work of documenting your impact.

comment by Ryan Kidd (ryankidd44) · 2024-10-02T18:09:14.903Z · LW(p) · GW(p)

1% are "Working/interning on AI capabilities."

Erratum: previously, this statistic was "7%", which erroneously included two alumni who did not complete the program before Winter 2023-24, which is outside the scope of this report. Additionally, two of the three alumni from before Winter 2023-24 who selected "working/interning on AI capabilities" first completed our survey in Sep 2024 and were therefore not included in the data used for plots and statistics. If we include those two alumni, this statistic would be 3/74 = 4.1%, but this would be misrepresentative as several other alumni who completed the program before Winter 2023-24 filled in the survey during or after Sep 2024.