Have You Tried Hiring People?

I strongly believe that, given the state of things, we really should spend way more on higher-quality people and see what happens.  Up to and including paying Terry Tao 10 million dollars [LW · GW]. I would like to emphasize I am not joking about this. 

I've heard lots of objections to this idea, and don't think any of them convince me it is not worth actually trying this. 

I have zero experience in hiring people; I only know how things seem from the opposite side (as a software developer). The MIRI job announcement seems optimized to scare people like me away. It involves a huge sunk cost in time -- so you only take it if you are 100% sure that you want the job and you don't mind the (quite real) possibility of paying the cost and being rejected at the end of the process anyway. If that is how MIRI wants to filter their applicant pool, great. Otherwise, they seem utterly incompetent in this aspect.

For starters, you might look at Stack Exchange and check how potential employees feel about being given a "homework" at an interview. Your process is worse than that. Because, if you instead invited hundred software developers on a job interview, gave them extensive homework, and then every single one of them ghosted you... at least you could get a suspicion that you were doing something wrong, and by experimenting you could find out what exactly it was. If you instead announce the homework up front, sure it is a honest thing to do, but then no one ever calls you, and you think "gee, no one is interested in working on AI alignment, I wonder why".

How to do it instead?

Here, my confidence is much lower, but it seems to me that you basically need to choose your strategy: do you have specific experts in mind, or are you just hoping to discover some talent currently unknown to you?

If you have specific experts in mind, you need to approach them actively. Not just post an announcement on your website, hope that they notice it, hope that they realize that you meant them, hope that they decide to jump through all the hoops, so that at the end you can tell them: "indeed, I had you in mind; I am glad you got the hints right -- now we can discuss salary and work conditions".

If you are looking for an unknown talent, you probably want some math and/or programming skills, so why not start with some mathematical or programming test, which would filter out the people without the skills. First round, publish the problems on LessWrong and let people type their answers into Google Forms. Yes, cheating is possible. That is why the second round will be done on Zoom, but only with those who passed the first round. Do not be too harsh at grading, just filter out the people who obviously do not have the talent. What is left is the people who have some talent in general, and are kinda interested in the job. Now you can start giving them more specific tasks and see how well they do. Maybe reward them by internet points first, so they can see how they compare to the competition. When the tasks get more difficult, start paying them real money, so that when it becomes too much of a time sink, it also becomes a side income. (Don't: "If you do this successfully, I will give you X money." Do: "Try your best, and I will give you X money for trying, as long as you give me something. Of course if the results are not up to our standards, we may at some moment decide to remove you from the list of participants.") At the end, either everyone fails, or you will have found your person. Even if everyone fails, you still get some data, for example at which step most of them failed.

Yes, there will be some wasted time and effort. It happens to all companies. Whenever you invite someone for an interview, and after an hour you decide that you don't want them for whatever reason, you just wasted one hour of salary of everyone who participated in that interview. That is a cost of hiring people.

Importantly: you need to have one person whose task is to do the recruitment. If you just expect it to happen, but it's actually no one's high priority, then... most likely it won't.

My general impression based on numerous interactions is that many EA orgs are specifically looking to hire and work with other EAs, many longtermist orgs are looking to specifically work with longtermists, and many AI safety orgs are specifically looking to hire people who are passionate about existential risks from AI. I get this to a certain extent, but I strongly suspect that ultimately this may be very counterproductive if we are really truly playing to win.

And it's not just in terms of who gets hired. Maybe I'm wrong about this, but my impression is that many EA funding orgs are primarily looking to fund other EA orgs. I suspect that a new and inexperienced EA org may have an easier time getting funded to work on a given project than if a highly experienced non-EA org would apply for funding to pursue the same idea. (Again, entirely possible I'm wrong about that, and apologies to EA funding orgs if I am mis-characterizing how things work. On the other hand, if I am wrong about this then that is an indication that EA orgs might need to do a better job communicating how their funding decisions are made, because I am virtually positive that this is the impression that many other people have gotten as well.)

One reason why this selectivity kind of makes sense at least for some areas like AI safety is because of infohazard concerns, where if we get people who are not focused on the long-term to be involved then they might use our money to do capability enhancement research instead of pursuing longtermist goals. Again, I get this to a certain extent, but I think that if we are really playing to win then we can probably use our collective ingenuity to find ways around this.

Right now this focus on only looking for other EAs appears (to me, at least) to be causing an enormous bottleneck for achieving the goals we are ultimately aiming for.

The objections myself and someone in the field of talent acquisition and retention received showed a failure to go meta. If the problem is not knowing how to turn money into talent then you hire for that. There is somewhere in the world someone whose time is very valuable who is among the best in the world at hiring. Pay them for their time.

This:

Is either a recruitment ad for a cult, or a trial for new members of an exclusive club. It's certainly not designed to intrigue what would be top applicants.

Could someone from MIRI step in here to explain why this is not being done? This seems like an extremely easy avenue for improvement.

If we expect pareto distribution to apply then the folks who will really move the needle 10x or more will likely need to be significantly smarter and more competent than the current leadership of MIRI. There likely is a fear factor, found in all organizations, of managers being afraid to hire subordinates that are noticeably better than them as they could potentially replace the incumbents, see moral mazes.

This type of mediocrity scenario is usually only avoided if turnover is mandated by some external entity, or if management owns some stake, such as shares, that increases in value from a more competent overall organization. 

Or of course if the incumbent management are already the best at what they do. This doesn’t seem likely as Eliezer himself mentioned encountering ’sparkly vampires’ and so on that were noticeably more competent.

The other factor is that now we are looking at a group that at the very least could probably walk into a big tech company or a hedge fund like RenTech, D.E. Shaw, etc., and snag a multi million dollar compensation package without a sweat, or who are currently doing so. Or likewise on the tenure track at a top tier school. 

>  Give Terrence Tao 500 000$ to work on AI alignement six months a year, letting him free to research crazy Navier-Stokes/Halting problem links the rest of his time... If money really isn't a problem, this kind of thing should be easy to do.

Literally that idea has been proposed multiple times before that I know of, and probably many more times many years ago before I was around.

What was the response?  (I mean, obviously it was "not interested", otherwise it would've happened by now, but why?)

[edit: my thoughts are mostly about AI alignment and research.  Some discussion seems to be about "EA", which likely DOES contain a bunch of underfunded "can probably buy it" projects, so I'd be surprised if there's a pile of unspent money unsure how to hire mosquito-net researchers or whatever. ]

This doesn't seem that surprising to me - the alignment problem in employees remains unsolved, and hiring people to solve AI Alignment seems one level abstracted from that.  IMO (from outside), MIRI is still in the "it takes a genius missionary to know what to do next" phase, not the "pay enough and it'll succeed" one.

I suspect that even the framing of the question exposes a serious underlying confusion.  There just isn't that much well-defined "direct work" to buy.  People in the in-crowd around here often call something a "technical problems" when they mean "philosophical problem requiring rigor".  Before you can assign work or make good use of very smart workers who don't join your cult, but just want a nice paycheck for doing what they're good at, you need enough progress on monitoring and measurement of alignment that you have any idea if they're helping or hurting your mission.

I've been involved in recruiting and hiring top-level (and mid-level and starting) technical and management talent at both big (FAANG) and medium (under 500 people) companies, and the single best strategy I know is to have lots of shovel-ready work for reasonably-well-paid entry- and mid-level employees, and over a few years some of those will evolve into stars.  Hiring existing stars works too, but requires the reputation and product that attracts them.  This ecosystem requires shockingly more sustainable throughput (revenue and self-funded growth) than I believe MIRI has any desire to create.  Almost certainly there isn't enough funding to make it work as a "pure expense" model.

Another common path is acquisition - find a company or group you want to hire, and overpay for their product in order to bring their key employees in.  If you're actually looking to spend $billions, this is probably the right answer - find people solving somewhat-similar technical problems, and combine them with your people solving the theoretical alignment problems.  


 

Even as someone generally interested in alignment while doing a job search, I had trouble seeing anything that'd work for me (though that is also partly because I live in Europe).

I commented on that thread, and also wrote this [LW · GW] post a few months ago, partially to express some thoughts on the announcement of Lightcone (& their compensation philosophy).  Recently I've even switched to looking for direct work opportunities [LW · GW] in the AI alignment sphere.

As is usual with ACX posts on the subject of AI alignment, the comments section has a lot of people who are totally unfamiliar with the space and are therefore extremely confident that various inadequacies (such as the difficulties EA orgs have in hiring qualified people) can easily be solved by doing the obvious thing.  This is not to say that everything they suggest is wrong.  Some of their suggestions are not obviously wrong.  But that's mostly by accident.

I agree that "pay people market rate" is a viable strategy and should be attempted. At the very least I haven't yet seen many strong arguments for what benefit is accrued by paying below-market rates (in this specific set of circumstances), and thus default to "don't shoot yourself in the foot".  I also think that most people are deeply confused about the kinds of candidates that MIRI (& similarly focused orgs) are looking for, in terms of researchers.  There are some problems with operating in a largely pre-paradigmatic field which impose non-trivial constraints on the kinds of people it even makes sense to hire:

• To the extent that you have "research directions", they're usually much less defined than "demonstrate a highly specific, narrow result at the edge of a well-established field"
• As a result, you mostly need incoming researchers to be self-sufficient or bootstrap to that state very quickly, because every hour spent on mentorship and guidance by an existing researcher is an hour not spent doing research (it's probably not a total loss, but it's not great)
• If you say "ok, bite the bullet, and spend 6 months - 2 years training interested math/cs/etc students to be competent researchers", they've definitely tried this on a smaller scale and my understanding is that it doesn't tend to work very well to produce the kinds of researchers who can make sustained progress on attacking relevant problems without so much help from existing researchers that it stops being worth it

So the general attitude present of "just hire a proper recruiter!" is missing all the ways in which that kind of suggestion is a total non-starter.  The kinds of candidates MIRI is looking for aren't legibly distinguished by their LinkedIn profiles.  Recruiters in general have niches, and (to the best of my knowledge) this is not a niche that is well-served by recruiters.  Superficially qualified candidates who don't actually believe in the problem that they're being paid to attack are probably not going to make much progress on it, but they will take up the time and attention of existing researchers.

Could MIRI do a better job of advertising their own job postings?  Maybe, unless what they're doing is effective at preferentially filtering out bad candidates without imposing an overwhelming barrier to good candidates.  MIRI is hardly the only player in the game at this point, though - Redwood, ARC, Anthropic, and Alignment AI are all hiring researchers and none of them (officially) require candidates to work their way up to being Alignment Forum regulars.

I've looked at these postings a few times, I kind of assumed that the organization was being run as a welfare program for members of a specific social club, that they were specifically excluding anyone who wasn't already central to that community, and that the primary hiring criteria was social capital.

So the perfect candidate is 'person already in this community with the highest social capital, and a compelling need for a means of support'.

OpenAI is an example of a project that threw a bunch of money at hiring people to do something about AI alignment. If you follow the perspective of Eliezer, they increased AI risk in the process.

You can easily go through LinkedIn and hire people to do alignment research. On the other hand, it's not easy to make sure that you are producing a net benefit on the resulting risk and not just speeding up AI development.

Intuitively, I expect that gathering really talented people and telling them to do stuff related to X isn't that bad of a mechanism for getting X done. The Manhattan Project springs to mind.

The Manhatten Project is not one that's famous for reducing X-risk. 

My 2 cents:

If you have high quality people working on a project, and you are already paying them "plenty of money," you can increase thier productivity by spending money to make sure they aren't wasting any of thier 24 hours/day doing things they don't want to do. People are reluctant to spend "thier own" money to hire a driver/maid/personal secretary/errand boy full time, but those are things that can be provided as an in-lind transfer. We know it works, because that's what Hollywood does: the star actors and directors have legions of assistants who anticipate and fulfil thier needs so the talent can focus on the task.

As of October, MIRI has shifted its focus. See their announcement [LW · GW] for details.

I looked up MIRI's hiring page and it's still in about the same state. This kind of makes sense given the FTX implosion. But I would ask whether MIRI is unconcerned with the criticism it received here and/or actively likes their approach to hiring? We know Eliezer Yudkowsky, who's on their senior leadership team and board of directors, saw this, because he commented on it.

I found it odd that 3/5 members of the senior leadership team, Malo Bourgon, Alex Vermeer, and Jimmy Rintjema, are from Ontario (Malo and Alex at least are alumns from University of Guelph). I think this is a relevant question given that the concern here is specifically about whether MIRI's hiring practices are appropriate. I am surprised about this both because the University of Guelph, as far as I know, is not particularly renowned as an AI or AI safety research institution, and because Ontario is physically distant from San Francisco, ruling out geographic proximity as an explanation.

A bit of Googling turned up MIRI's own announcement page for Malo Bourgon's hiring as COO (he's now CEO). "Behind the scenes, nearly every system or piece of software MIRI uses has been put together by Malo, or in a joint effort by Malo and Alex Vermeer — a close friend of Malo’s from the University of Guelph who now works as a MIRI program management analyst."

I would like to understand better what professional traits made Malo originally seem like a good hire, both given that his background doesn't sound particularly AI or AI safety-focused. "His professional interests included climate change mitigation, and during his master’s, he worked on a project to reduce waste through online detection of inefficient electric motors. Malo started working for us shortly after completing his master’s in early 2012, which makes him MIRI’s longest-standing team member next to Eliezer Yudkowsky."

I'd also like to know what professional traits led to the hire of Alex Vermeer, given that both Alex and Malo were hired in 2012. Was a pre-existing friendship a factor in the hire, and if so, to what extent?

The three people from Ontario seem particularly involved in the workshop/recruiting/money aspect of the organization:

• "Malo’s past achievements at MIRI include: coordinating MIRI’s first research workshops and establishing our current recruitment pipeline." (From the hiring announcement page)
• For another U Guelph alumn listed on their team page, "Alex Vermeer improves the processes and systems within and surrounding MIRI’s research team and research programs. This includes increasing the quality and quantity of workshops and similar programs, implementing best practices within the research team, coordinating the technical publication and researcher recruiting pipelines, and other research support projects."
• "Jimmy Rintjema stewards finances and regulatory compliance, ensuring that all aspects of MIRI’s business administration remain organized and secure." (This is also from the team page)

In my personal opinion, Eliezer's short response to this post and the lack of other response (as far as I can see here) suggest that MIRI may be either uninterested or incapable of managing its perception and reputation, at least on and around LessWrong. That makes me wonder how well it can realistically fulfill its new mission of public advocacy. I am also curious to know in detail how it came to be that so many people from Ontario occupy positions in senior leadership?

Yes, I agree with you.  I didn't bring it up in the original comment because I think the primary bottleneck is not just "insufficient # of people who are smart enough to do this work", but the combination of "insufficient # of people who smart enough, epistemically grounded enough, motivated enough, and capable-of-novel-thoughts enough".  My main point is that there are plenty of people who pass the bar on g-factor but fail one or more of the other criteria.

Eliezer already addressed this in his writing, no?  He used to think that we were in the "Manhattan Project" phase where throwing gobs of money at the problem would make something happen.  Then he realized he was mistaken and we aren't nearly that far along in rigorously understanding reasoning.  We're at the phase where if you need to be told what to do/work-on/think-about it'd be hard to provide that direction (at least in the employee or contract sense).  It's unclear to me if 10-15 years of progress have changed that, but it seems like mostly no; ML has had some very impressive feats, but why any of it works remains elusive.

On the other hand, FWIW I'm a software engineer who's intersted in this research and been following whatever he can for the last decade+, but any path from hobby to actually working on this seems extremely costly or outright nonviable.  Do a bunch of networking (already costly for introverts), much of which involves travel, to maybe land an internship in one of the most expensive cities in the US where you can fight for whatever scraps of housing are available.  I don't know for sure if I could actually contribute, but I know with filters like this we aren't likely to find out.

For what its worth. I went to Miri's Summer fellow research program thing. And if there was a permanent version of that, I would definitely be applying. I'm on a PhD course because I didn't see any big "Want to work on alignment research" posters.

Generally, I agree that exploring more effective ways to scale things up is a good idea - including radical reassessment of recruitment.

However, there's a fundamental point that ought to be borne in mind when making arguments along the lines of [structures like X solve problems well]:

1. We did not choose this problem.
2. We don't get to change our minds and pick another problem if it looks difficult/impossible.

[There's a sense in which we chose the AGI problem, and safety/alignment is just a sub-problem that comes up, but I don't think that's a sensible take on the situation: super-human adversarial agency is not a sub-problem like any other in history]

Companies choose problems that look solvable, and if a particular sub-problem turns out to be much harder than expected they'll tend to find workarounds or to abandon the problem and pick one that looks more approachable. On the highest level, a company only needs to make money; conveniently, that is a problem with many known solutions.

The same applies to universities and individuals: the most productive ones focus on problems that seem solvable. When they bump into the kind of problems that have a significant chance of being unsolvable, most will pivot to something else (and most who don't, fail).

Paying a load of smart people is a good way to ensure that some problems get solved.
Paying a load of smart people is a good way to solve [some particular problem you chose] when you chose a suitable problem.

That's not the situation here: we must solve this particular fixed problem, and it's a problem no sane person would choose to need to solve.

Again, I agree with considering taking some radical steps (including throwing money at the problem) to get the right people working on this. However, my argument for that would be entirely from first principles.

Any argument based around "This is what successful companies/universities/people do" implicitly assumes that the metric for 'success' usefully translates. I don't think it does - unless you have a reference class for successful approaches to [extremely hard problems organisations had no choice but to solve] (noting that "they had no choice" is usually a huge exaggeration).

For example, even the Manhattan project isn't a great comparison, since it was set up precisely because the problem looked difficult-but-solvable.

We're in an unprecedented situation, so any [what usually works] approach can only hope to address sub-problems (the kind of sub-problems that [what usually works] will find). We still need some expectation that the solution to such sub-problems will help solve our initial problem.

If people are curious about MIRI's workshops and general recruiting process, see this post [LW · GW] by someone who went through the process and was rejected, with long comments by people involved in hiring at MIRI. (Note that this is all pre-pandemic.)

Speaking to the value of secrecy and how this disincentivizes the make-it-prestigious|hire-lots-of-talent path, perhaps the X-risk family of EA should throw money at mechanism design experts and research people from DARPA or the intelligence community for proposals on building highly secure research orgs, then throw money at talent under the good-secrecy org.

It isn't as though "we need you to help stop the AI apocalypse" is that different a pitch from "we need you to help stop the bioweapon apocalypse" or "we need you to help stop the nuclear apocalypse" pitches the government has been using since WW2. I would rate the latter two as mostly successful on the secrecy front, evaluated independently from effectiveness.