Discussion about AI Safety funding (FB transcript)
post by Akash (akash-wasil) · 2023-04-30T19:05:34.009Z · LW · GW · 8 comments8 comments
Comments sorted by top scores.
comment by porby · 2023-05-01T00:45:18.228Z · LW(p) · GW(p)
Ways to turn $$$ into impact that aren't happening:
...
4. Increasing independent alignment researcher salary to like 150k/year (depending on location) to enable better time money tradeoffs.
This is a bit surprising to me!
If we're talking about reasonably proven high-skill researchers, 150,000 USD/year in a reasonable cost of living area (e.g. DFW area) is... still sufficiently uncompetitive that some people that could have contributed will definitely follow the money instead. With the implied skills and background, it's not too hard to make a lot more money with less effort.
The implication that current funding tends to target even lower salaries than this is concerning.
Random thoughts:
- If this number is a rough average that includes junior researchers (e.g. just out of school), there might be no issue.
- I know funding at or above 150k/year is possible based on the public reports, so there isn't a hard threshold, which is good.
- I could see an argument for preferring funding those who are already sufficiently deep that they are willing to take significant paycuts relative to industry because of risk/timelines/charitability/autonomy. That subset of people might have higher impact per dollar spent (and have lower rates of fraud), but I doubt that this is a survival maximizing strategy unless you have enough talent in that pool that you become funding constrained.
- I'm worried that the rate of funding across the field is lower than it should be (given my timelines, at least). Seems like the game's going to end with a lot of unspent resources.
↑ comment by Steven Byrnes (steve2152) · 2023-05-01T12:49:34.269Z · LW(p) · GW(p)
I think a lot of people applying to do independent alignment research do live in expensive areas like SFBay but are on the younger side and don’t have kids and are willing to share a cramped apartment with roommates etc. Basically, the same kind of people who might alternatively choose to go to grad school despite equally pathetically low stipends.
When I was applying for my first independent alignment research grant in 2020, by contrast, I had daycare expenses and a mortgage and so on. I made a massive spreadsheet and calculated that I needed $150k/yr to make it work, so that’s what I asked for. This was a substantial pay cut from my industry job, and I still felt weird / guilty / something-or-other because I had the strong impression that I was asking for like 3-5× more money than were most people applying for the same kind of grant. But whenever I brought that up explicitly to the people in the field who were helping me with grant-applications etc., they all took great pains to assure me that it was fine—I should ask for an amount that would work for my situation, and funders can always say no, but anyway they’re probably paying more attention to the project quality than the cost at these scales. (And anyway, those same funders are also probably donating to nonprofits with comparable or higher cost-per-employee.) Anyway, I did find a funder [LW · GW]! :) :)
Replies from: porby↑ comment by porby · 2023-05-02T19:52:47.522Z · LW(p) · GW(p)
I still felt weird / guilty / something-or-other because I had the strong impression that I was asking for like 3-5× more money than were most people applying for the same kind of grant. But whenever I brought that up explicitly to the people in the field who were helping me with grant-applications etc., they all took great pains to assure me that it was fine
Good to hear you got funding at that rate, and that this is explicitly considered good-and-normal! That was the vibe I got (including from my own grant getting funded), and was part of why I was surprised in the first place.
↑ comment by DusanDNesic · 2023-05-01T11:44:11.593Z · LW(p) · GW(p)
Perhaps not all of them are in the Bay Area/London? 150k per year can buy you three top professors from Eastern European Universities to work for you full time, and be happy about it. Sure, other jobs pay more, but when unconstrained from living in an expensive city, these grants actually go quite far. (We're toying with ideas of opening research hubs outside of most expensive hubs in the world, exactly for that reason)
Replies from: thomas-larsen, porby↑ comment by Thomas Larsen (thomas-larsen) · 2023-05-01T22:13:37.667Z · LW(p) · GW(p)
Fwiw I'm pretty confident that if a top professor wanted funding at 50k/year to do AI Safety stuff they would get immediately funded, and that the bottleneck is that people in this reference class aren't applying to do this.
There's also relevant mentorship/management bottlenecks in this, so funding them to do their own research is generally a lot less overall costly than if it also required oversight.
(written quickly, sorry if unclear)
↑ comment by porby · 2023-05-02T20:16:53.125Z · LW(p) · GW(p)
It can indeed go far in lower cost of living areas- if the average salary is brought down by a bunch of willing and highly effective cheap talent, that would be perfectly fine and good. (And I endorse hubs in cheaper areas! I might have moved to SF, if not for it being... SF.)
I do still worry about practical competitiveness in this case, though. For reference, housing in the DFW area is 3-5x cheaper than the bay area, so 150k/year buys you quite a bit of luxury... but you can find work for even more than that. A lot more, depending on specialty and experience. If we model researchers as simple economic agents, offers need to compete with other offers, not just the cost of living.
Those top professors might have reasons to not take higher paying (in terms of real pay vs. cost of living) industry jobs. Maybe they don't want to move internationally, maybe they've got family, maybe they like the autonomy their current position has, maybe they believe in the cause sufficiently that they view the pay cut as a form of charity, and so on. In terms of funding strategy, though, I wouldn't want to rely on people accepting dramatically lower rates than they can demand.
Replies from: DusanDNesic↑ comment by DusanDNesic · 2023-05-03T23:07:22.811Z · LW(p) · GW(p)
I agree, but AIS jobs are usually fairly remote-friendly (unlike many corporate jobs) and the culture is better than in most universities that I've worked with, so it has many non-wage perks. Question is, can people in cheap cost-of-living places find such high paid work? In Eastern Europe, usually no - there are other people willing/able to work for less so all wages are low, cost of living correlates with wages in that sense too. So giving generous salaries to experts that are in/are willing to relocate to lower cost of living places is cost-effective, insofar as they are currently an underutilized group. I know in EE there are people who would make for good researchers, but are unaware of the problems, salary landscape and such, which is something we're trying to fix (and global awareness of AI is helping a lot).
comment by Zian · 2023-05-01T06:13:58.691Z · LW(p) · GW(p)
The content may be public but does copyright law allow these sorts of quotes?
Which venue controls Facebook posts' copyright? I assume that Lesswrong.com's liability is also affected by the law that applies to the location of Lightcone's office.
I do not know the answer but desired to mention them in case they are relevant.