Donating to MIRI vs. FHI vs. CEA vs. CFAR

I'm convinced AGI is much more likely to be built by a government or major corporation, which makes me more inclined to think movement-building activities are likely to be valuable, to increase the odds of the people at that government or corporation being conscious of AI safety issues, which MIRI isn't doing.

MIRI's AI workshops get outside mathematicians and AI researchers involved in FAI research, which is good for movement building within the population of people likely to be involved in creating an AGI.

Eliezer on Facebook:

There seems to have been a very lopsided flow of funds into the MIRI and CFAR fundraisers. The balance is tilted sufficiently that I'm now willing to call it for the next marginal dollar being more valuable at CFAR than at MIRI, at least until CFAR's fundraiser completes.

(Of course that only applies to the current fundraisers, not to the question in general.)

I've heard that CFAR is already trying to move in the direction of being self-sustaining by charging higher fees and stuff. I went to a 4-day CFAR workshop and was relatively unimpressed; my feeling about CFAR is that they are providing a service to individuals for money and it's probably not a terrible idea to let the market determine if their services are worth the amount they charge. (In other words, if they're not able to make a sustainable business or at least a university-style alum donor base out of what they're doing, I'm skeptical that propping them up as a non-alum is an optimal use of your funds.)

FHI states that they are interested in using marginal donations to increase the amount of public outreach they do. It seems like FHI would have a comparative advantage over MIRI in doing outreach, given that they are guys with PhDs from Oxford and thus would have a higher level of baseline credibility with the media, etc. So it's kind of disappointing that MIRI seems to be more outreach-focused of the two, but it seems like the fact that FHI gets most of its funding from grants means they're restricted in what they can spend money on. FHI strikes me as more underfunded than MIRI, given that they are having to do a collaboration with an insurance company to stay afloat, whereas MIRI has maxed out all of their fundraisers to date. (Hence my decision to give to FHI this year.)

If you do want to donate to MIRI, it seems like the obvious thing to do would be to email them and tell them that you want to be a matching funds provider for one of their fundraisers, since they're so good at maxing those out. (I think Malo would be the person to contact; you can find his email on this page.)

I think it's worth considering the funding landscape and the funding situations of the various organisations - "how much good" is obviously not static, and will depend on factors including this one. I've commented in other places in this thread that FHI is underfunded relative to its needs as an organisation and the research we think is important to do. I believe this is true.

I also believe that FHI has funding opportunities available to it that aren't available to CEA/CFAR/MIRI. For example, at present ~80% of our overall funding comes from a combination of industry and academic grants.

This has its downsides - while this allows us scope to do a large amount of pure existential risk research, some of our research has to be tailored towards satisfying our funders, and so from a pure X-risk reduction point of view we are somewhat constrained. There are also a number of valuable uses of money that we can't use these funds for.

It means the ~20% that we gain from philanthropic sources has very high utility. These are the funds that allow us to do public outreach, thesis prize competitions, further philanthropic/industry fundraising, extensive networking and visitor programmes, and other important work that doesn't fit into a traditional academic research project - thus these funds add utility to the other ~80% of funds. Also, we have one researcher currently funded from a philanthropic donation, and he has the freedom to work purely on the Xrisk work he (or Nick Bostrom) considers most important, rather than having to devote some time to e.g. framing research output to fit an academic funder. This unconstrained research capability is of obvious value.

However, it also means that we can (at least at present) get the bulk of the funds needed to run the FHI from sources that may not be available to these other valuable organisations. While we can't usually get grants to get exactly what we want to do, we can usually get grants to do something reasonably close - enough so that we can use them to get useful work done (even if we lose some efficiency compared to unconstrained funding). Hence, while I think donations sent to FHI will do quite a lot of good, I would be very reluctant to see too much funding diverted away from organisations that may not have the same breadth of funding options available to them if at all possible. (Frankly, I prefer to get funding from companies that sponsor rugby teams when they're not sponsoring FHI ;) )This is also a reason CSER's been targeting academic councils in the first instance.

tl;dr: We get quite a lot of our funding from other sources, and so may not always need additional funds as much as organisations with a less broad funding profile. That said, philanthropic funds that come to us play an important role in our running and have high utility.

Nick, Anders and Toby have been consulted by government agencies in the past, particularly Nick has done that several times (even by the Thailand's government apparently). If your concern is influence over government, FHI wins, given I don't think movement building would get us as far as having a Prime Minister meeting with FHI staff. It would have to be one serious movement to match just one or two meetings. It's likely that there aren't even enough eligible brains for a "AI-risks movement" of such scale.

However, it is not the case "influence over the government" should be the most important criteria. Mainly, because right now we wouldn't even know exactly what to tell them, and it might take decades until we do. Hence, the most important criteria is level of basic research. The mere fact your question hasn't any clear answers means we need more basic research, and thus that MIRI/ FHI have preference. I couldn't know whether FHI or MIRI would be better. As a wild guess, I would say FHI does more research but that it somehow feeds from MIRI non-academic flexibility and movement building. Likely, whichever had the preference over resources would lose this preference relatively fast as it outgrew the other.

On the other hand, I have heard MIRI/FHI/CEA staff claiming they are much more in need of qualified people than money. So, if CFAR is increasing qualification then they ought to have priority. But it's not clear if they are really doing that yet.

movement-building activities are likely to be valuable, to increase the odds of the people at that government or corporation being conscious of AI safety issues

CEA and CFAR don't do anything, to my knowledge, that would increase these odds, except in exceedingly indirect ways. FHI might be the most credible opportunity here because of their academic associations, which give them more credibility in PR. I remember Luke saying that FHI and CSER's academic ties as the reason why they -- an not MIRI -- are better suited to do publicity than FHI.

Therefore, while I disagree with you that the most important thing is to increase the odds of the people at that government or corporation being conscious of AI safety issues, I think that given what values you have told me, FHI is the most likely to maximize them.

We've collated a list of all the approaches that seem to be on the table in the effective altruism community for improving the long-run future. There's some other options, including funding GiveWell and GCRI. This doc also explains a little more of the reasoning behind the approaches. If you like more detail on how 80k might help reduce the risk of extinction, drop me an email at ben@80000hours.org.

In general, I think the question of how best to improve the long-run future is highly uncertain, but has high value of information, so the most important activities are: (i) more prioritisation research (ii) building flexible capacity which can act on whatever turns out to be best in the future.

MIRI, FHI, GW, 80k, CEA, CFAR, GCRI all aim to further these causes, and are mutually supporting, so are particularly hard to disentangle. My guess is that if you buy the basic picture, the key issues will be things like 'which organisation has the most pressing room for more funding at the moment?' rather than questions about the value of the particular strategies.

Another option would be to fund research into which org can best use donations. There's a chance this could be commissioned through CEA, though we'll need to think of some ways to reduce bias!

Disclaimer: I'm the Executive Director of 80,000 Hours, which is part of CEA.

Seems like a case of harder choices matter less.

Perhaps Luke Muehlhauser thinks CFAR is universe-scale useful for the same reason Eliezer Yudkowsky does?

And remember: To be a PC, you’ve got to involve yourself in the Plot of the Story. Which from the standpoint of a hundred million years from now, is much more likely to involve the creation of Artificial Intelligence or the next great advance in human rationality (e.g. Science) than… than all that other stuff. Sometimes I don’t really understand why so few people try to get involved in the Plot. But if there’s one thing I’ve learned in life, it’s that the important things are accomplished not by those best suited to do them, or by those who ought to be responsible for doing them, but by whoever actually shows up.

(If this is indeed the whole reason EY thinks so, besides movement-building).

Argh! Original post didn't go through (probably my fault), so this will be shorter than it should be:

First point:

I know very little about CEA, and a brief check of their website leaves me a little unclear on why Luke recommends them, aside from the fact that they apparently work closely with FHI.

CEA = Giving What We Can, 80,000 Hours, and a bit of other stuff

Reason -> donations to CEA predictably increase the size and strength of the EA community, a good proportion of whom take long-run considerations very seriously and will donate to / work for FHI/MIRI, or otherwise pursue careers with the aim of extinction risk mitigation. It's plausible that $1 to CEA generates significantly more than $1's worth of x-risk-value [note: I'm a trustee and founder of CEA].

Second point:

Don't forget CSER. My view is that they are even higher-impact than MIRI or FHI (though I'd defer to Sean_o_h if he disagreed). Reason: marginal donations will be used to fund program management + grantwriting, which would turn ~$70k into a significant chance of ~$1-$10mn, and launch what I think might become one of the most important research institutions in the world. They have all the background (high profile people on the board; an already written previous grant proposal that very narrowly missed out on being successful). High leverage!

If I were a talented but broke EA, I would try to convince my mom to invest the money in me, since that would make me a more effective philanthropist faster.

A lot of people here argue that microdonations make a large whole, but I'm not sure they're applying probabilistic reasoning to a huge amount of donations actually happening. Or maybe the donation is huge, what do I know.

Is there an update on this issue? Representatives from nearly all the relevant organizations have stepped in, but what's been reported has done little to resolve my confusion on this issue, and I think of myself as divided on it as Mr. Hallquist originally was. Dr. MacAskill, Mr. O'Haigeartaigh, Ms. Salamon have all provided explanations for why they believe each of the organizations they're attached are the most deserving of funding. The problem is that this has done little to assuage my concern about which organization is in the most need of funds, and will have the greatest impact given a donation in the present, relative to each of the others.

Thinking about it as a write this comment, it strikes me an unfortunate case of events when organizations who totally want to cooperate towards the same ends are put in the awkward position of making competing(?) appeals to the same base of philanthropists. This might have been mentioned elsewhere in the comments, but donations to which organization do any of you believe would lead to the biggest return of investment in terms of attracting more donors, and talent, towards existential risk reduction as a whole? Which organization will increase the base of effective altruists, and like individuals, who would support this cause?